ArticlePDF Available

Efficient optimization of the dual-index policy using Markov chains

Authors:

Abstract and Figures

We consider the inventory control of a single product in one location with two supply sources facing stochastic demand. A premium is paid for each product ordered from the faster 'emergency' supply source. Unsatisfied demand is backordered and ordering decisions are made periodically. The optimal control policy for this system is known to be complex. For this reason we study a type of base-stock policy known as the dual-index policy (DIP) as control mechanism for this inventory system. Under this policy ordering decisions are based on a regular and an emergency inventory position and their corresponding order-up-to-levels. Previous work on this policy assumes deterministic lead times and uses simulation to find the optimal order-up-to levels. We provide an alternate proof for the result that separates the optimization of the DIP in two one-dimensional problems. An insight from this proof allows us to generalize the model to accommodate stochastic regular lead times and provide an approximate evaluation method based on limiting results so that optimization can be done without simulation. An extensive numerical study shows that this approach yields excellent results for deterministic lead times and good results for stochastic lead times.
Content may be subject to copyright.
Efficient Optimization of the Dual-Index Policy Using Markov Chains
Joachim Arts
1,
, Marcel van Vuuren
2
, Gudrun Kiesm¨uller
3
1
Eindhoven University of Technology, School of Industrial Engineering
P.O. Box 513, 5600 MB Eindhoven, the Netherlands
2
Consultants in Quantitative Methods,
P.O. Box 414, 5600 AK Eindhoven, the Netherlands
3
Christian-Albrechts-University at Kiel
Olshausenstr 40, 24098 Kiel, Germany
September 2, 2010
Abstract
We consider the inventory control of a single product in one location with two supply sources facing
stochastic demand. A premium is paid for each product ordered from the faster ‘emergency’ supply
source. Unsatisfied demand is backordered and ordering decisions are made periodically. The optimal
control policy for this system is known to be complex. For this reason we study a type of base-stock
policy known as the dual-index policy (DIP) as control mechanism for this inventory system. Under
this policy ordering decisions are based on a regular and an emergency inventory position and their
corresponding order-up-to-levels. Previous work on this policy assumes deterministic lead times and
uses simulation to find the optimal order-up-to levels. We provide an alternate proof for the result that
separates the optimization of the DIP in two one-dimensional problems. An insight from this proof allows
us to generalize the model to accommodate stochastic regular lead times and provide an approximate
evaluation method based on limiting results so that optimization can be done without simulation. An
extensive numerical study shows that this approach yields excellent results for deterministic lead times
and good results for stochastic lead times.
Keywords: inventory, dual sourcing, dual-index policy, Markov Chain, approximation, lead times, D/G/c/c queue
1. Introduction
Research into inventory systems is mostly done under the assumption that only one supplier or
supply mode exists to procure, manufacture or ship goods. While many useful results have been
obtained under this assumption (e.g. News-vendor type results for many systems, see van Houtum
2006 for an overview), these models nevertheless omit an important aspect of many real inventory
systems, namely that inventories can be replenished in more than one way. For example, it is
common that one item can be procured from different suppliers or manufactured in different plants.
Corresponding author, E-mail: j.j.arts@tue.nl
1
Alternatively an item may be shipped over sea or by air (expediting). Even within the production
environment of a single plant the production lead time can be decreased by producing in overtime.
In all these examples there are multiple ways to replenish inventory with different lead times and
costs.
The situations described in the previous subparagraph can be approached in roughly two ways.
The first approach is to carefully select one of the suppliers/supply modes and then source all
inventory from that supplier/supply mode. We refer to the problem of making this decision as
vendor selection. The second approach is to use both suppliers/supply modes simultaneously. This
paper is concerned with the latter approach which we refer to as dual sourcing.
Suppliers are becoming more willing to offer different supply modes to their customers. Pac-
car parts in Eindhoven, for example, which handles spare-part logistics for DAF trucks, makes a
distinction between regular and emergency delivery modes for shipping parts to different locations
throughout Europe. Another situation where multiple supply modes occur naturally is in remanu-
facturing systems. In this setting serviceable products can be produced from raw materials or by
remanufacturing returned items. These two modes of inventory replenishment are naturally asso-
ciated with different costs and lead times. A similar situation also occurs in the inventory control
of spare parts. Spare parts are kept on stock so that a capital good can readily be made available
upon failure of a part. The failed part is then sent into normal or emergency repair with associated
different lead times and costs.
In this paper we study a general model for the inventory control in dual sourcing systems.
We consider the inventory control of a single product in one location that is reviewed periodically
and has two supply sources with different lead times. The lead times are assumed to be integer
multiples of the review period. The faster supply source will be referred to as the emergency
supplier while the slower supply source will be called the regular supplier. Units procured from the
emergency supplier incur additional per unit cost. Ordering from the regular channel may represent
manufacturing somewhere in Asia, while ordering through the emergency channel may represent
ordering from a more expensive local supplier. Other applications include, but are not limited to,
shipping goods by sea (‘regular’) or air (‘emergency’) freight and manufacturing with (‘emergency’)
or without (‘regular’) overtime. The problem we shall consider is the minimization of holding and
ordering costs subject to a service level constraint.
Models for the situations described above are difficult to analyze. Under specific restrictive as-
sumptions, such as the assumption of a unit lead time difference, the analysis can become tractable.
2
For this latter model the optimal policy is known but the application area is very narrow. When
lead time differences are more than one period the optimal policy is known to be complex, difficult
to implement and computationally hard to obtain (Whittmore & Saunders 1977, Feng et al. 2006a
and Feng et al. 2006b). In this paper we investigate exactly this context. For this reason we
consider a class of base-stock type policies and optimize within this class. Specifically we study the
dual-index policy (DIP) that has the attractive property of reducing to the optimal policy when
the lead time difference is only one period. This policy was originally proposed for remanufacturing
systems (Kiesm¨uller, 2003). It is easily implementable and has been shown to perform very close to
the optimal policy (Veeraraghavan & Scheller-Wolf 2008). Numerical studies have also shown that
in general it outperforms competing heuristic policies such as the simpler single-index policy and
constant order policy (Scheller-Wolf et al. 2003, Klosterhalfen et al. 2010). The good performance
of the DIP is also highlighted by Sheopuri et al. (2010) who benchmark their heuristic policies
against “the best available heuristic in the literature - the best dual index policy”. Until now the
DIP has resisted analytical or even approximate analytical optimization so researchers have had to
resort to simulation based procedures.
The DIP policy tracks two inventory positions: a regular inventory position (on-hand stock
+ all outstanding orders - backlog) and an emergency inventory position (on-hand stock + out-
standing orders that will arrive within the emergency lead time - backlog). In each period ordering
decisions are made to raise both inventory positions to their order-up-to-levels. Under this policy
the emergency inventory position can, and indeed often does, exceed its corresponding order-up-to-
level. This excess is called the overshoot and plays a central role in the analysis of the DIP. Despite
its relatively simple form, optimization of the DIP still requires substantial computational effort
because it requires determining several overshoot distributions. In principle the overshoot distribu-
tion can be obtained exactly by solving a multidimensional discrete time Markov chain (DTMC).
However, this approach suffers from the curse of dimensionality and consequently the usual ap-
proach is to determine the overshoot distribution by simulation. Veeraraghavan & Scheller-Wolf
(2008) prove a separability result that drastically decreases the amount of simulation needed, but
the computational time remains substantial.
In this paper we study a model comparable to that of Veeraraghavan & Scheller-Wolf (2008).
The main differences are that we (i) consider the minimization of holding and purchasing costs
subject to a service level constraint and (ii) generalize the model by incorporating stochastic regular
lead times. The experience of the authors is that service level constraints are very common in
3
practice and are usually found in the form of service level agreements. The use of a service level
constraint in our model is convenient for managers who would like to incorporate service level
agreements in their decision making or who would like to assess the impact of different service level
agreements on their operations. In any case Van Houtum & Zijm (2000) show that models with
penalty costs for disservice (e.g. backlogging cost) and service level constraints are related so that
if one problem can be solved, the other one can be too.
For our service level model the ability to determine the overshoot distribution remains key in
evaluating and optimizing the DIP. We provide an alternate proof of the aforementioned separa-
bility result for both deterministic and stochastic lead times. An insight from this proof is used
to construct a one-dimensional DTMC that describes the overshoot process. By approximating
the transition probabilities for this DTMC based on limiting results we obtain a computationally
efficient optimization procedure.
This paper is organized as follows. In Section 2 we review the literature on dual sourcing and
position our results with respect to earlier work. We then present the model with deterministic
lead times in Section 3 and introduce the dual-index policy formally. In section 4 we analyze this
policy and give limiting results to easily find approximately optimal settings. Next we generalize
our model to accommodate stochastic regular lead times in Sections 5 and 6. Section 7 provides
an extensive numerical study on the accuracy of our approximation. We give conclusions and
directions for further research in Section 8.
2. Literature review
Minner (2003) provides a review of the literature pertaining to many different issues surrounding
multiple supply sources. Broadly speaking the research in multiple sourcing is divided into the
strategic approach, which studies issues such as exchange rate volatility, risk management and
vendor selection, and the operational approach that mainly studies the inventory control of such
systems. Among the different perspectives we focus on operational/tactical control of multiple
sourcing systems. One body of research focusses on the number of supply sources as a decision
variable and usually assumes that different sources are identical. In these situations replenishment
orders are split among the different supply sources and optimal order splitting is the object of study.
Another body of research considers situations with two (or more) suppliers that have different
lead times. Replenishing inventory from the faster supplier incurs additional cost. This paper
4
contributes to this body of research. As Minner (2003) provides an excellent review of research
up to around 2001 we will discuss key results from before that time only briefly. Then we discuss
relevant research since that time.
Early research focusses on the structure of the optimal policy for periodic inventory systems with
dual sourcing. Barankin (1961) considers the single period problem with instantaneous emergency
delivery and a regular lead time of one period. Fukuda (1964) formulates the problem as one of
negotiable lead time for the infinite horizon case and gives an analytical derivation of the optimal
policy by discounted dynamic programming. He considers a system that operates in discrete time,
and has two suppliers whose lead times are deterministic and differ by exactly one period. Sethi
et al. (2003) extend Fukuda’s (1964) model with fixed ordering costs, demand forecast updates
and show that the optimal policy is of the (s, S)-type. Yazlali & Erhun (2009) extend Fukuda’s
(1964) model with minimum and maximum capacity requirements for both suppliers and derive
the optimal policy. Scheller-Wolf & Tayur (2009) add order bands and state dependent demand
to Fukuda’s model and derive the optimal policy. The assumption that the lead times of both
suppliers differ by only one period is crucial to obtaining optimal policies with a simple structure.
In 1977 Whittmore & Saunders and more recently Feng et al. (2006a, 2006b) showed that in the
optimal policy ordering decisions depend on the entire vector of outstanding orders for general lead
time differences. Thus the optimal policy is complex and not of the base-stock type when the lead
time difference is more than one period.
Despite the fact that the optimal policy for general lead time differences has been known to be
complex since 1977, the focus on good policies with a simpler structure is rather recent. Scheller-
Wolf et al. (2003) consider the same setting as Whittmore & Saunders (1977) and propose the
single-index policy under which ordering for both the emergency and regular supplier are based on
a single state parameter: the inventory position. This policy is simple and can easily be optimized
when demand distributions are mixtures of Erlangian distributions. When the lead time difference
is one period the single-index policy also reduces to the optimal policy. Kiesm¨uller (2003) proposes
the use of a policy that tracks two inventory positions associated with different lead times in the
context of a remanufacturing system. The key idea here is that the decision on the amount to order
at the emergency supplier should not be based on information about orders that will arrive after
this order. Veeraraghavan & Scheller-Wolf (2008) study this policy in the context of two supplier
models. They provide the aforementioned separability result for deterministic lead times. This
separability result separates the optimization of the DIP, which is a two-dimensional optimization
5
problem, to two one-dimensional optimization problems.
Sheopuri et al. (2010) study generalizations of the dual-index policy and relate these generaliza-
tions to the lost sales inventory problem. The generalization of the dual-index policy comes at the
expense of policy structure in that the generalized policy is no longer of the base-stock type for both
regular and emergency orders. These policies outperform the dual-index policy by on average 1.1%
in their computational study, and require roughly the same amount of computational effort. They
also consider a type of policy that is base-stock in the sense that each period exactly the demand of
the previous period is ordered but the allocation between the regular and the emergency supplier is
heuristic and cannot be parameterized. In a computational study they show that the performance
of these policies is similar to the performance of the dual-index policy but less computationally
burdensome compared to the simulation-optimization procedure they use to find a good dual-index
policy. We present an efficient procedure to optimize the dual-index policy that can also be used
to optimize the generalized dual-index policy that Sheopuri et al. (2010) propose.
A completely different policy for this problem setting is the standing order or constant order
policy. In this policy the regular supplier delivers a fixed quantity every period while the emergency
supplier may be controlled using various types of policies. This type of policy was first studied by
Rosenshine & Obee (1976). Recent contributions in this area are Chiang (2007), who derives the
optimal policy structure given that the regular order quantity is fixed and Allon & van Mieghem
(2008), who approximate the related Tailored Base Surge policy using Brownian motion.
A closely related problem is the expedition of orders after they have entered the pipeline. Lawson
& Porteus (2001) study this problem in a serial multi-echelon periodic review context. They show
that a type of base-stock policy, called a “top down base-stock policy” is optimal when orders can
be expedited and delayed at will in the entire supply chain. Zhou and Chao (2010) consider a
model similar to that of Lawson and Porteus (2001) and derive the optimal policy when lead time
differences between regular and expedited shipping are again restricted to one period. They also
provide newsvendor bounds on policy parameters. Gallego et al. (2007) study a single stock-point
in continuous time with the possibility of expediting existing orders and derive the optimal policy
under the assumption of Poisson demand.
In assemble-to-order systems, where lack of a single component may render the system unable
to fill an order, expediting is often also included in the model (Plambeck & Ward 2007, Hoen et al.
2010). In this setting expediting is usually assumed to have zero lead time except for one period
problems (Fu et al. 2010). Arslan et al. (2001) provide analytic models for when and how to
6
expedite in make-to-order systems. Thus this model does not include inventory.
All literature in dual sourcing assumes deterministic lead times except for Song & Zipkin (2009)
and Gaukler et al. (2008). Song and Zipkin study a model of a stock-point facing Poisson demand
operating in continuous time. They assume a (S 1, S)-type ordering policy and show how to
model this system as a network of queues with one or more overflow bypasses. Gaukler et al.
(2008) also consider a single stock-point operating in continuous time and propose a policy based
on the classical (Q, R)-policy. They show how to find optimal parameter settings under a set of
specific assumptions.
The setting we consider is similar to the settings in Fukuda (1964), Whittmore & Saunders
(1977) and Veeraraghavan & Scheller-Wolf (2008). Our two most important contributions are (i)
the development of an efficient approximation for the overshoot distribution so that optimization of
the DIP becomes computationally more feasible and (ii) the incorporation of stochastic lead times
in the periodic review setting.
3. Model with deterministic lead times
Our model is similar to the model studied by Veeraraghavan & Scheller-Wolf (2008). We consider
the inventory control of a single product in one location with two supply sources facing stochastic
demand. The purchase price for regular (emergency) units is c
r
(c
e
). Note that since in the long
run all demand must be ordered, incurring at least c
r
per unit, we are only interested in those
units for which we pay more than c
r
. Thus we may say a premium c = c
e
c
r
is paid for each
product ordered from the faster ‘emergency’ supply source, and ignore the constant of expected
demand times c
r
. Unsatisfied demand is backordered and ordering decisions are made periodically.
Without loss of generality we assume the length of a review period to be one. Demand per period
is a sequence of non-negative i.i.d. discrete random variables {D
n
} with n a period index. We
assume that Pr(D > 0) > 0 and Pr(D < ) = 1. The net inventory (stock on-hand - backlog)
at the beginning of period n will be denoted I
n
. Any on-hand stock I
+
n
at the beginning of a
period n incurs a holding cost of h per SKU. (We use the standard notations x
+
= max(0, x) and
x
= max(0, x)). We denote the backlog at the beginning of a period B
n
= I
n
. Orders placed
at the regular (emergency) channel arrive after a deterministic lead time l
r
(l
e
) and we assume
l := l
r
l
e
, l 1. Lead times are assumed to be an integer multiple of the review period. The
regular (emergency) order placed in period n is denoted Q
r
n
(Q
e
n
). Later (in Section 5) we will
7
Figure 1: Graphical representation of model with deterministic regular lead times
relax the assumption that l
r
is deterministic. A schematic representation of the situation described
above is given in Figure 3.
As control mechanism for this inventory system we study the dual-index policy (DIP), defined
by two parameters (S
e
, S
r
), which operates as follows. At the beginning of each period n we review
the emergency inventory position
IP
e
n
= I
n
+
P
nl
i=nl
r
Q
r
i
+
P
n1
i=nl
e
Q
e
i
(1)
and if necessary place an emergency order Q
e
n
to raise the emergency inventory position to its
order-up-to-level S
e
,
Q
e
n
= (S
e
IP
e
n
)
+
. (2)
After placing the emergency order we inspect the regular inventory position which includes the
emergency order just placed
IP
r
n
= I
n
+
P
n1
i=nl
r
Q
r
i
+
P
n
i=nl
e
Q
e
i
= IP
e
n
+ Q
e
n
+
P
n1
i=n+1l
Q
r
i
(3)
and place a regular order Q
r
n
to raise the regular inventory position to its order-up-to-level S
r
,
Q
r
n
= S
r
IP
r
n
. (4)
After ordering, shipments are received and demand for the period is satisfied or backordered if
there is no stock available. Thus within a period n the sequence of events can be summarized as
follows: (1) review the on-hand inventory and incur holding costs hI
+
n
; (2) review the emergency
inventory position and place an emergency order Q
e
n
; emergency ordering costs are incurred as
cQ
e
n
; (3) review the regular inventory position and place a regular order Q
r
n
; (4) receive shipments
Q
e
nl
e
and Q
r
nl
r
; (5) demand D
n
occurs and is satisfied except for possible back-orders B
n
. Note
that the emergency inventory position under this policy can, and indeed often does, exceed the
emergency order-up-to-level S
e
. The amount by which the emergency inventory position exceeds
8
the emergency order-up-to-level is called the overshoot. After ordering the emergency inventory
position is given by S
e
+ O
n
where O
n
{0, 1, ..., S
r
S
e
} is the overshoot and satisfies
O
n
= IP
e
n
+ Q
e
n
S
e
= (IP
e
n
S
e
)
+
. (5)
Determining the stationary distribution of the overshoot O will play a key role in evaluating the
performance of a given policy (S
r
, S
e
).
Our objective is the minimization of the long run average cost subject to a modified fill-rate
constraint. The modified fill-rate is defined as
γ = 1 E[B]/E[D]. (6)
The modified fill-rate is closely related to the regular fill-rate often denoted β. As opposed to
the regular fill-rate, orders staying in backlog multiple time periods affect the modified fill-rate
negatively for the entire time they are backlogged. Van Houtum and Zijm (2000) show that γ
service level models are related to backlogging costs for fairly general inventory systems. They also
provide a discussion of different service levels in inventory systems. When service-levels are high,
the modified fill-rate is a very tight lower bound of the regular fill-rate (Van Houtum 2006).
The average costs related to our problem are the costs of emergency ordering and holding costs
given by
C(S
e
, S
r
) = hE[I
+
] + cE[Q
e
]. (7)
We are now in a position to formulate the optimization problem P:
(P) min C(S
e
, S
r
)
s.t. γ(S
e
, S
r
) γ
0
S
e
, S
r
Z.
(8)
Here γ
0
denotes the target service level. This problem is a non-linear integer programming problem
(NLIP). The integrality constraint on S
e
and S
r
is the consequence of the discrete nature of demand.
Note that continuous demand distributions can also be used, but discretization has to be applied.
If the integrality constraint on the decision variables are relaxed fractional values, i.e. randomized
solutions may become optimal. In practice however fractional solutions are awkward and our
optimization method will require integrality of the involved decision variables. An overview of all
introduced notations and other notations that will be introduced in later sections is given in Table
3.
9
notation description
A
n
Amount of ordered products that will not arrive within the emergency lead time
in period n after ordering (:=
P
n
i=n+1l
Q
r
i
)
B
n
Backlog in period n
c Premium to buy one product at the emergency supplier
C(S
e
, S
r
) Average holding and incremental ordering costs for policy (S
e
, S
r
)
γ Modified fill rate
γ
0
Target modified fill-rate
D
n
Demand in period n, random variable
Difference between regular and emergency order-up-to-level (:= S
r
S
e
)
h Inventory holding cost per period per SKU
I
n
The net inventory (on-hand stock - backlog) at the beginning of period n
IP
r
n
Regular inventory position at the beginning of period n after ordering at the emergency supplier
IP
e
n
Emergency inventory position at the beginning of period n before ordering
l
e
Replenishment lead time for emergency orders
l
r
Replenishment lead time (deterministic) for regular orders
l Difference between regular and emergency replenishment lead time (:= l
r
l
e
)
n Period index
O
n
Overshoot in period n (:= (IP
e
n
S
e
)
+
)
S
r
Regular order-up-to-level
S
e
Emergency order-up-to-level
Q
r
n
Regular order quantity placed in period n
Q
e
n
Emergency order quantity placed in period n
Table 1: Summary of notations
4. Analysis of model with deterministic lead times
This section is organized as follows. In section 4.1 we present our separability result and show how it
can be exploited to find the optimal DIP if the overshoot distribution can be determined. In section
4.2 we present an exact one-dimensional Discrete Time Markov Chain (DTMC) that describes the
overshoot. Following this, in section 4.3 we provide approximations for the transition probabilities
such that this DTMC can be efficiently utilized to approximate the overshoot distribution.
Throughout the analyses in this paper for any random variable X
n
we define the stationary ex-
pectation and distribution as E[X] = lim
n→∞
1
n
P
n
i=1
X
n
and Pr(X x) = lim
n→∞
1
n
P
n
i=1
I{X
n
x} where I{x} is the indicator function of the event x. Whenever we drop the index of a random
variable we are referring to the stationary random variables with mean and distribution defined
above. Additionally we denote the k-fold convolution of a random variable X as X
(k)
and the
squared coefficient of variance of a random variable X as c
2
X
:=
Var[X]
E
2
[X]
.
10
4.1 Optimization
In our analysis we shall see that the difference between S
r
and S
e
plays an important role. Therefore,
we define := S
r
S
e
. This definition allows for the specification of a DIP as either (S
e
, S
r
) or
(S
e
, S
e
+ ∆). For the analysis it will be more convenient to consider S
e
and ∆ as decision variables.
In this section we show how to find the optimal S
e
for fixed ∆. This allows for a simple search
procedure over to find the optimal DIP.
First we investigate an interesting property of the DIP. Consider the pipeline stock that will
not arrive within the emergency lead time and denote this quantity A
n
in period n after ordering:
A
n
=
P
n
i=n+1l
Q
r
i
. (9)
Since we will consider a more general model in section 5, the proofs of the next three lemma’s will
be deferred to section 6.
Lemma 4.1. (Key functional relation) Consider the dual-index policy for a system with determin-
istic lead times and A
n
as defined in equation (9). Suppose that IP
r
k
S
r
for some k N
0
. Then
for all n k the dual-index policy ensures that the following identity holds
∆ = O
n
+ A
n
. (10)
The intuition behind this lemma is simple. After ordering the regular inventory position is
S
r
, but it also equals the emergency inventory position (S
e
+ O
n
) plus all outstanding orders not
included in the emergency inventory position (A
n
). Thus S
e
+ ∆ = S
e
+ O
n
+ A
n
. Lemma 4.1
essentially states that A
n
and O
n
are complements so that any knowledge regarding A
n
implies
knowledge regarding O
n
. The identity = O
n
+ A
n
also completely describes the operation of
the DIP as is evident from the proof. Before establishing our separability result we need one more
lemma which is originally due to Veeraraghavan & Scheller Wolf (2008).
Lemma 4.2. (Recursions for O
n
, Q
e
n
and Q
r
n
) Consider the model with deterministic lead times
operated by the dual-index policy. The overshoot O
n
, emergency order quantity Q
e
n
and regular
order quantity Q
r
n
satisfy the following recursions:
O
n+1
= (O
n
D
n
+ Q
r
n+1l
)
+
, (11)
Q
e
n+1
= (D
n
O
n
Q
r
n+1l
)
+
, (12)
Q
r
n+1
= D
n
Q
e
n+1
. (13)
11
The recursions (11)-(13) are quite intuitive. Equation (11) describes that the overshoot dimin-
ishes each period with the demand and increases with the regular order that enters the information
horizon of the emergency inventory position. The emergency order quantity can also be thought of
as the ‘undershoot’, i.e., Q
e
n
= (S
e
IP
e
n
)
+
from which relation (12) follows. Relation (13) follows
from the property that in each period the total order amount equals the demand in the previous
period. With these results we now establish the separability result, part of which also appears
as proposition 4.1 in Veeraraghavan & Scheller-Wolf (2008). We remark again that our proof is
different.
Lemma 4.3. (Separability result) Consider the model with deterministic lead times operated by
the dual-index policy. The distributions of O, Q
r
and Q
e
depend on S
r
and S
e
only through their
difference ∆ = S
r
S
e
.
Let us define O
as the stationary random variable O for a given ∆. Lemma 4.3 can be exploited
to obtain the optimal DIP for fixed ∆.
Theorem 4.4. (On the optimal choice for S
e
) Consider the dual-index policy for the control of our
model with deterministic lead times. For fixed the optimal S
e
is the smallest integer that satisfies
the following inequality:
X
k=0
E
D
(l
e
+1)
S
e
k
+
Pr(O
= k) (1 γ
0
)E(D). (14)
Proof. As a consequence of lemma 4.3 the cost term related to emergency ordering, cE[Q
e
], becomes
a fixed constant when is fixed. Thus, for fixed the relevant cost function is given by
e
C(S
e
) =
hE[I
+
] and the problem reduces to a one-dimensional optimization problem we shall call Q.
(Q) min
e
C(S
e
)
s.t. γ(S
e
, S
e
+ ∆) γ
0
S
e
Z.
(15)
Now by the identity γ = 1 (E[B]/E[D]) the service level constraint can be modified into a
constraint on E[B]. The expected backlog can be found by conditioning on the emergency inventory
position after ordering, using that demand is an i.i.d. sequence and recalling that by lemma 4.3
the distribution of O is already fixed:
E[B] =
X
k=0
E
D
(l
e
+1)
S
e
k
+
Pr(O
= k) (1 γ
0
)E[D]. (16)
12
The objective function
hE[I
+
] =
X
k=0
E
S
e
+ k D
(l
e
+1)
+
Pr(O
= k) (17)
is non-decreasing in S
e
as can easily be shown by recalling that probabilities are non-negative and
using finite differences. This implies that the smallest integer S
e
that satisfies inequality (16) is the
optimal solution to Q, which completes the proof.
Remark It is also easy to show that E[B] is a non-increasing function of S
e
. Thus the optimal S
e
given can easily be found using a simple method such as a bisection search.
The above result provides a simple way to find the optimal DIP if the distribution of O and
E[Q
e
] can be determined for a fixed ∆. If this can be done one may simply perform a search
procedure over to find the globally optimal DIP. To evaluate the cost term cE[Q
e
] for the
objective function of problem P we note that the first moment of O completely determines the first
moment of Q
e
through the relations E[Q
r
] =
E[A]
l
=
E[O]
l
and E[D] = E[Q
r
] + E[Q
e
]. Thus from
the distribution of O it is easy to determine the cost term cE[Q
e
]. In the next two subsections we
describe a one-dimensional Discrete Time Markov Chain (DTMC) that describes the overshoot.
Moreover, we provide approximations for its transition probabilities which enable the overshoot to
be approximated efficiently.
4.2 A one-dimensional Markov Chain for the Overshoot
Lemma 4.1 gives insight into the behavior of O
n
. Instead of studying O
n
we may study A
n
which
has a straightforward physical interpretation as the pipeline stock that will not arrive within the
short lead time l
e
. A
n
obeys the following recurrence relation:
A
n+1
= O
n+1
=
D
n
P
n
i=n+2l
Q
r
i
+
=
D
n
A
n
+ Q
r
n+1l
+
= min(∆, A
n
Q
r
n+1l
+ D
n
).
(18)
The first equality follows from lemma 4.1, the second by simultaneous substitution of equations (11)
and (10). The third equality follows again by substituting equation (10) while the forth is a standard
identity. In principle A
n
can be modeled by a DTMC. To construct this DTMC for A
n
however,
we would need to store the last l regular order quantities in the state information. This leads to
13
an l-dimensional Markov Chain. From equation (18) we retrieve that Q
r
n
{0, 1, ..., } and so
this DTMC would have
P
k=0
P
k
x
1
=0
P
kx
1
x
2
=0
···
P
k
P
l1
i=1
x
i
x
l
=0
k
x
1
,x
2
,...,x
l
states. It is computationally
infeasible to find the equilibrium distribution of this DTMC for most practical instances. To remedy
this we study a more compact DTMC with only one dimension and + 1 states.
Observe that the recurrence relation (18) completely defines a Markov Chain for A
n
if the
probability mass functions of D and {Q
r
n+1l
|A
n
} are known. This DTMC is defined by the
transition probabilities p
ij
= Pr (A
n+1
= j|A
n
= i) that can be obtained by distinguishing the
cases j < and j = ∆. First consider the case j < ∆, we have
p
ij
= Pr(A
n+1
= j|A
n
= i)
= Pr(A
n
Q
r
n+1l
+ D
n
= j|A
n
= i)
=
P
j
k=0
Pr(Q
r
n+1l
= A
n
+ D
n
j|A
n
= i, D
n
= k) Pr(D
n
= k)
=
P
j
k=0
Pr(Q
r
n+1l
= i + k j|A
n
= i) Pr(D = k).
(19)
The case j = ∆ is very similar:
p
i
= Pr(A
n+1
= ∆|A
n
= i)
= Pr(A
n
Q
r
n+1l
+ D
n
|A
n
= i)
=
P
i
k=0
Pr(D
n
+ Q
r
n+1l
A
n
|A
n
= i, Q
r
n+1l
= k) Pr(Q
r
n+1l
= k|A
n
= i)
=
P
i
k=0
Pr(Q
r
n+1l
= k|A
n
= i) Pr(D + k i).
(20)
Now we organize these transition probabilities in the transition matrix P :
P =
p
00
. . . p
0∆
.
.
.
.
.
.
.
.
.
p
∆0
. . . p
∆∆
. (21)
If we let π(x) denote Pr(A = x), π = [π(0), . . . , π(∆)] and e = [1, 1, . . . , 1]
T
, then the stationary
distribution π can be found by solving the set of linear equations
πP = π, πe = 1. (22)
The distribution of D is assumed to be known, but the distribution of {Q
r
n+1l
|A
n
} is in fact
unknown. In the next subsection we construct an approximation for this distribution based on
limiting results so that the introduced one-dimensional DTMC can be used to approximate the
overshoot distribution.
14
4.3 Approximations for the transition probabilities
To determine the transition probabilities in the DTMC of the previous section we need the proba-
bility mass functions of D and {Q
r
n+1l
|A
n
}. The latter can be approximated using the following
(limiting) result.
Proposition 4.5. The following statements hold:
(i) As , Pr(Q
r
n+1
= x) Pr(D
n
= x).
(ii) As , Pr
Q
r
n+1l
= x|A
n
= y
Pr
D
n+1l
= x|
P
n
i=n+1l
D
i
= y
.
(iii) For ∆ = 1, Pr
Q
r
n+1l
= x|A
n
= y
= Pr
D
n+1l
= x|
P
n
i=n+1l
D
i
= y
.
Proof. Part (i) and (ii) are special cases of proposition 6.5; we defer the proof to there. Part (iii)
holds trivially under the conditioning A
n
= 0. For the conditioning A
n
= 1 we need only show
that Pr(Q
r
n+1l
= 1|A
n
= 1) = Pr(D
n+1l
= 1|
P
n
i=n+1l
D
i
= 1) because Pr(Q
r
n+1l
= 0|A
n
= 1)
is the complement of Pr(Q
r
n+1l
= 1|A
n
= 1). Recall the definition of A
n
as the sum of l regular
orders. When A
n
= 1, there is exactly one order of one SKU in the pipeline A
n
. Since is
an upperbound to the number of items in A
n
this one item blocks other items from entering the
pipeline as it travels through. Hence this one item can be any of the regular orders included in A
n
with probability 1/l, i.e., Pr(Q
r
n+1l
= 0|A
n
= 1) = 1/l. Now we have
Pr
D
n+1l
= 1|
P
n
i=n+1l
D
i
= 1
=
Pr(D = 1) Pr
D
(l1)
= 0
Pr(D
(l)
= 1)
=
Pr(D = 1) Pr(D = 0)
l1
l Pr(D = 1) Pr(D = 0)
l1
(23)
= 1/l.
But 1/l = Pr(Q
r
n+1l
= 0|A
n
= 1) as required.
Intuitively parts (i) and (ii) of proposition 4.5 are obvious because ∆ = corresponds to single
sourcing with the regular supplier, in which case Q
r
n+1
= D
n
. Parts (ii) and (iii) of proposition 4.5
suggest that Pr
D
n+1l
= x|
P
n
i=n+1l
D
i
= y
can be used to approximate Pr
Q
r
n+1l
= x|A
n
= y
as this approximation is exact for extremely small (∆ = 1) and extremely large (∆ ).
Thus an approximation for Pr
Q
r
n+1l
= x|A
n
= y
is given by
Pr
Q
r
n+1l
= x|A
n
= y
Pr
D
n+1l
= x|
P
n
i=n+1l
D
i
= y
(24)
=
Pr(D = x) Pr
D
(l1)
= y x
Pr
D
(l)
= y
.
15
Using this approximation for Pr
Q
r
n+1l
= x|A
n
= y
we can compute an approximation for Pr(A =
x) by solving the set of linear equations (22). Then by using relation (10) we obtain an approxi-
mation for the distribution of O as Pr(O = x) = Pr(A = ∆ x).
Remark Note that the above approximation is also exact when l = 1, because in this situation
A
n
= Q
r
n+1l
= Q
r
n
. Since the DIP is the optimal policy for l = 1 this approach yields the globally
optimal policy whenever l = 1.
Numerical experiments indicate that this approximation works well in a wide range and closely
approximates both the first two moments of O and the shape of its distribution. The shape of
the overshoot distribution for four typical examples are given in Figure 4.3, where the overshoot
distribution as determined by simulation is shown in conjunction with the approximation based on
the above analysis. The demand distributions in these examples are either mixtures of geometric or
mixtures of negative binomial distributions fitted on the moments given. The fitting procedure is
due to Adan et al. (1996) and details are provided in the appendix. Results of a detailed numerical
study are presented in Section 7.
5. Model with stochastic lead times
The model we consider is identical to the model described in section 3 with one exception: now we
assume that L
r
n
is a stochastic integer and that the lower bound of its support is at least l
e
+ 1.
This ensures that the emergency supply mode is always faster than the regular supply mode; this
makes physical sense and lends tractability to the model. We only assume the regular supply mode
to be stochastic because the stochasticity for the emergency supply mode is usually negligible. For
example the emergency supply mode may represents overnight shipping using a logistics service
provider such as FedEx. To facilitate analysis we define the random variable L
n
as
L
n
:= L
r
n
l
e
. (25)
The support of L is constituted by the positive integers N. We can think of L
r
n
as consisting of a
deterministic part l
e
and a stochastic part L
n
. We will assume that {L
n
} is an i.i.d. sequence and
Pr(L = ν) = q
ν
. This implies that order crossover is possible and places us in a setting similar to
that of Robinson et al. (2001). Further we let l
n
and l
r
n
denote realizations of the random variables
L
n
and L
r
n
.
16
Figure 2: Overshoot distributions for a few illustrative cases as determined by simulation and the
corresponding Markov Chain approximations
Inventory positions are now defined using set notation. Let X
n
be the set of all period indices
such that at the beginning of period n before ordering, the regular orders from these periods have
not yet arrived in stock,
X
n
= {k|k n l
r
k
, k < n}.
Additionally let Y
n
be the set of all period indices such that at the beginning of period n before
ordering the regular orders from these periods have not yet arrived in stock but will do so within
the emergency lead time:
Y
n
= {k|k n l
r
k
, k n l
k
}.
Using these sets we can again define the emergency and regular inventory positions as
IP
e
n
= I
n
+
P
iY
n
Q
r
i
+
P
n1
i=nl
e
Q
e
i
(26)
17
and
IP
r
n
= I
n
+
P
iX
n
Q
r
i
+
P
n
i=nl
e
Q
e
i
= IP
e
n
+ Q
e
n
+
P
iX
n
\Y
n
Q
r
i
.
1
(27)
Notice that these definitions reduce to the earlier definitions in case of deterministic regular
lead times. We also remark that we do not necessarily need to know the realizations of L
r
n
up to
time n for the inventory positions to be well defined. The only necessary information needed is
to know in real time when the order from period k will arrive within the emergency lead time l
e
,
i.e., we need to know when k = n l
k
. In essence the random variable L
r
n
consists of a random
component L
n
and a deterministic component l
e
. We assume that the random component becomes
known before or at the time the remaining lead time of a regular order is l
e
.
There are multiple ways for this information to become available in practice. First we may
know what the regular lead time will be as soon as we place an order. Second we may know when
a regular order will arrive within the emergency lead time because this time is naturally associated
with known events such as a shipment harboring at the port. In the context of manufacturing in
overtime or other use of flexible capacity this information may be available by simple inspection of
the job floor.
Ordering decisions are still given by equations (2) and (4) and the overshoot still satisfies the
original definition in equation (5). We now proceed to analyze the DIP when regular lead times
are stochastic.
6. Analysis of model with stochastic lead times
Our analysis will proceed along the same lines as the analysis for deterministic regular lead times,
i.e., we show how to find the optimal DIP for fixed (Section 6.1) and provide a one dimensional
DTMC that describes the overshoot (Section 6.2). We provide approximations for the transition
probabilities of this DTMC in Section 6.3.
6.1 Optimization
Let us turn again to the amount of pipeline stock that will not arrive within the emergency lead
time, A
n
. Let U
n
be the set of all period indices such that in period n after ordering the regular
orders from these periods will not arrive within the emergency lead time:
U
n
= {k|k n l
k
+ 1, k n}. (28)
1
A\B is the set A minus B
18
Now the definition of A
n
can be written as
A
n
=
P
iU
n
Q
r
i
. (29)
Lemma 6.1. (Key functional relation) Consider the model with stochastic lead times operated by
the dual-index policy and A
n
as defined in equation (29). Suppose that IP
r
k
S
r
for some k N
0
.
Then for all n k the dual-index policy ensures that the following identity holds
∆ = O
n
+ A
n
. (30)
Proof. Reconsider the regular inventory position as given in equation (27),
IP
r
n
= IP
e
n
+ Q
e
n
+
P
iX
n
\Y
n
Q
r
i
. (31)
Now we substitute the definition of the overshoot (from equation (5)) and add Q
r
n
to both sides of
this equation,
IP
r
n
+ Q
r
n
= S
e
+ O
n
+
P
iX
n
\Y
n
∪{n}
Q
r
i
. (32)
By supposition IP
r
n
S
r
so Q
r
n
= S
r
IP
r
n
and the left-hand side of (32) becomes S
r
. When we
take a closer look at the set over which the sum in (32) runs it is straightforward to verify that
U
n
= X
n
\Y
n
{n} so that we can substitute the definition of A
n
to obtain
S
r
= S
e
+ O
n
+ A
n
. (33)
Rearrangement and substitution of the identity ∆ = S
r
S
e
yields the result.
Lemma 6.1 is a direct generalization of lemma 4.1 and essentially states that A
n
and O
n
are
direct compliments also in the presence of stochastic regular lead times. Note also that lemma 6.1
holds for all ergodic stochastic processes {L
n
}
nN
0
, not just i.i.d. sequences.
Now we introduce V
n
the set of period indices such that at the beginning of period n after
ordering the regular orders from these periods will enter the information horizon of the emergency
inventory position in period n + 1,
V
n
= {k|k = n l
k
+ 1}. (34)
We emphasize that the sets X
n
and Y
n
are defined before ordering while U
n
and V
n
are defined
after ordering. As before we now turn our attention to recursions for O
n
, Q
e
n
and Q
r
n
and then
establish our separability result.
19
Lemma 6.2. (Recursions for O
n
, Q
e
n
and Q
r
n
) Consider the model with stochastic lead times as
defined in Section 5. The overshoot O
n
, emergency and regular order quantities satisfy the following
recursions:
O
n+1
=
O
n
D
n
+
P
iV
n
Q
r
i
+
, (35)
Q
e
n+1
=
D
n
O
n
P
iV
n
Q
r
i
+
, (36)
Q
r
n+1
= D
n
Q
e
n+1
. (37)
Proof. The emergency inventory position satisfies
IP
e
n+1
= IP
e
n
+ Q
e
n
D
n
+
P
iV
n
Q
r
i
= S
e
+ O
n
D
n
+
P
iV
n
Q
r
i
.
(38)
Rewriting the definition of the overshoot (equation (5)) we obtain
O
n+1
= (IP
e
n+1
S
e
)
+
= (S
e
+ O
n
D
n
+
P
iV
n
Q
r
i
S
e
)
+
= (O
n
D
n
+
P
iV
n
Q
r
i
)
+
.
(39)
Similarly for the emergency order quantity we have by rewriting (2):
Q
e
n+1
= (S
e
IP
e
n+1
)
+
= (D
n
O
n
P
iV
n
Q
r
i
)
+
.
(40)
The identity Q
r
n+1
= D
n
Q
e
n+1l
follows immediately from the fact that the DIP ensures that in
each period the total amount ordered equals demand from the previous period.
With these results we can prove the same separability result that was shown to hold for deter-
ministic regular lead times.
Lemma 6.3. (Separability result) Consider the model with stochastic lead times as defined in
Section 5. The distributions of O and Q
e
and Q
r
depend on S
r
and S
e
only through their difference
∆ = S
r
S
e
.
Proof. Recall the recursions in lemma 6.2. To make these equations independent of the start-
ing conditions we substitute the identity for O
n
in lemma 6.1. This substitution also makes the
operation of the DIP explicit:
O
n+1
=
D
n
P
iU
n
\V
n
Q
r
i
+
, (41)
Q
e
n+1
=
D
n
+
P
iU
n
\V
n
Q
r
i
+
, (42)
Q
r
n+1
= D
n
Q
e
n+1
. (43)
20
For the summation
P
iU
n
\V
n
Q
r
i
we read 0 whenever U
n
\V
n
= . These recursions completely
determine the stochastic processes {O
n
}, {Q
r
n
} and {Q
e
n
} once the stochastic sequences {D
n
},
and {L
n
} have been specified. Since the stochastic processes {O
n
} and {Q
e
n
} and {Q
r
n
} can be
described completely using S
r
and S
e
only through their difference, it follows that their stationary
distributions are functions of S
r
and S
e
only through their difference.
Remark In establishing lemma 6.3 we did not require that either {D
n
} or {L
n
} are i.i.d. sequences.
In principle the stationary overshoot distribution is well defined when is fixed for all ergodic
processes {D
n
} and {L
n
} such that D
n
N
0
and L
n
N
0
for all n N
0
. We do use that {D
n
}
and {L
n
} are i.i.d. in sections 6.2 and 6.3 to construct an efficient approximation for Pr(O = x).
However the distribution of O, Q
e
or Q
r
can be determined by simulation for more general processes
{D
n
} and/or {L
n
}.
Let us define O
as the stationary random variable O for a given ∆. Lemma 6.3 leads to the
following theorem on the optimal choice for S
e
for fixed
Theorem 6.4. (On the optimal choice for S
e
) Consider the model with stochastic lead times as
defined in Section 5. For fixed the optimal S
e
is the smallest integer that satisfies the following
inequality
X
k=0
E
D
(l
e
+1)
S
e
k
+
Pr(O
= k) (1 γ
0
)E(D). (44)
Proof. The proof is analogous to the proof of Theorem 4.4 and therefore omitted.
The optimal DIP for the system with stochastic lead times can also be found by a search
procedure over ∆. To find the cost term cE[Q
e
] for the objective function of problem P in this more
general situation, we make use of the identities E[Q
r
] =
E[O]
E[L]
and E[D] = E[Q
r
] + E[Q
e
]. In the
next two sections we describe a one-dimensional DTMC and transition probability approximations
for our generalized model.
6.2 A one-dimensional Markov Chain for the overshoot
As was the case for the model with deterministic lead times, lemma 6.1 allows us to study A
n
to
find the distribution of O. A
n
still has the appealing physical interpretation as the pipeline stock
21
that will not arrive within the short lead time l
e
and obeys the following recurrence relation
A
n+1
= O
n+1
=
D
n
P
iU
n
\V
n
Q
r
i
+
=
D
n
A
n
+
P
iV
n
Q
r
i
+
= min
, A
n
P
iV
n
Q
r
i
+ D
n
(45)
It is evident from the model with discrete lead times that an exact DTMC for this problem suffers
even more from the curse of dimensionality. For this reason we again turn our attention to a one-
dimensional DTMC that can be constructed in a manner analogous to that in Section 4.2. This
DTMC is given by the transition probabilities p
ij
= Pr (A
n+1
= j|A
n
= i):
p
ij
=
P
j
k=0
Pr
P
iV
n
Q
r
i
= i + k j|A
n
= i
Pr(D = k), if j < ∆;
P
i
k=0
Pr
P
iV
n
Q
r
i
= k|A
n
= i
Pr(D + k i), if j = ∆.
(46)
To make this one-dimensional DTMC of use, it remains to find the distribution of
P
iV
n
Q
r
i
|A
n
or an approximation thereof. This will be done in the next subsection.
6.3 Approximations for the transition probabilities
To determine the transition probabilities in the DTMC of the previous section we need the proba-
bility mass functions of D and
P
iV
n
Q
r
i
|A
n
. The latter can be approximated using the following
limiting result.
Proposition 6.5. The following statements hold
(i) As , Pr(Q
r
n+1
= x) Pr(D
n
= x)
(ii) As , Pr
P
iV
n
Q
r
i
= x|A
n
= y
Pr
P
n
i=n−|V
n
|+1
D
i
= x|
P
n
i=n−|U
n
|+1
D
i
= y
Proof. We rewrite equation (45) to
A
n+1
= min
, A
n
P
iV
n
Q
r
i
+ D
n
= min
,
P
iU
n
Q
r
i
P
iV
n
Q
r
i
+ D
n
= min
,
P
iU
n
\V
n
∪{n+1}
Q
r
i
Q
r
n+1
+ D
n
= min
, A
n+1
Q
r
n+1
+ D
n
.
(47)
Now if we let and recall the condition Pr(D < ) = 1 we immediately retrieve part (i) of
the proposition. Part (i) also implies that D
n
dist
= Q
r
n+1
for all n when , where
dist
= denotes
22
equality in distribution. Now since {D
n
} is i.i.d., so is {Q
r
n
} when . This implies that as
, the distribution of
P
iV
n
Q
r
i
dist
= Q
r
(|V
n
|)
dist
= D
(|V
n
|)
and A
n
dist
= Q
r
(|U
n
|)
dist
= D
(|U
n
|)
. From
this part (ii) immediately follows.
Remark When considering deterministic lead times we already showed in proposition 4.5 that the
approximation we propose is exact also for = 1. For stochastic L
n
this is no longer the case.
The numerical results in Section 7 reflect this fact.
Part (ii) of proposition 4.5 suggests that Pr
P
n
i=n−|V
n
|+1
D
i
= x|
P
n
i=n−|U
n
|+1
D
i
= y
can be
used to approximate Pr
P
iV
n
Q
r
i
= x|A
n
= y
. The computation of this approximation is however
not straightforward because it requires knowledge of the random variables |U
n
| and |V
n
| which in
turn depend on the process {L
n
}. Indeed for the computation of this probability we digress to
study the joint stationary distribution of |U
n
| and |V
n
| when L
n
is assumed to be a sequence of i.i.d
random variables with finite support. In principle one may study the joint distribution of |U
n
| and
|V
n
| for different lead time processes {L
n
}.
Let K
n
denote the number of orders in the pipeline that will not arrive within the emergency
lead time in period n after ordering,
K
n
= |U
n
|. (48)
Further let Λ
n
denote the number of orders that are about to enter the information horizon of the
emergency inventory position,
Λ
n
= |V
n
|. (49)
We wish to determine the joint stationary distribution of these two quantities Pr(K = κ Λ =
λ). We do this recursively. Recall that the distribution of L is given by q
ν
= Pr(L = ν), ν
{1, 2, ..., L
max
}. Further we define
ϕ
κ,λ,ν
= Pr(K = κ Λ = λ|orders were placed the last ν periods only (not before)), (50)
where we allow for orders of size 0 (which may occur if demand in a certain period is zero).
Obviously, this definition means that the distribution needed is given by
Pr(K = κ Λ = λ) = ϕ
κ,λ,L
max
:= ϕ
κ,λ
, (51)
since orders that were placed more than L
max
periods ago cannot belong to the sets U
n
or V
n
. The
probabilities ϕ
κ,λ,ν
can be computed recursively as follows:
ϕ
κ,λ,ν
= ϕ
κ111
q
ν
+ ϕ
κ1,λ,ν1
·
P
L
max
m=ν+1
q
m
+ ϕ
κ,λ,ν1
·
P
ν1
m=1
q
m
. (52)
23
The initial probabilities are straightforwardly seen to be
ϕ
1,0,1
=
P
L
max
m=2
q
m
, ϕ
1,1,1
= q
1
, ϕ
κ,λ,1
= 0 otherwise. (53)
This concludes our derivation of the joint stationary distribution of |U
n
| and |V
n
|.
Remark The process K
n
can also be thought of as the number of customers in a discrete time
D/G/L
max
/L
max
-queue. Each period n a customer arrives (order is placed) and that customer
immediately enters service for a random time L
n
(order stays in the set U for L
n
periods). Thus
this D/G/c/c-queue has the special property that the service distribution has a finite support
on {1, ..., L
max
} while the interarrival time is 1. In general the evaluation of the steady state
distribution of D/G/c/c-queues cannot be done in polynomial time if it can be done at all. For
this specific case the evaluation can be done in O(L
4
max
) time. To see this, note that the number
of times we compute recursion (52) including the initialization before we obtain ϕ
κ,λ
is given by
P
L
max
+1
i=2
P
i
x=2
x =
P
L
max
+1
i=2
i
2
+i2
2
=
1
2
P
L
max
+1
i=2
i
2
+
1
2
P
L
max
+1
i=2
i L
max
=
1
6
L
3
max
+ L
2
max
+
5
6
L
max
.
(54)
Since computing recursion (52) can be done in O(L
max
) time, the overal complexity is O(L
4
max
).
Now that the joint distribution of K and Λ is known, we can compute the approximation for
Pr
P
iV
n
Q
r
i
= x|A
n
= y
by conditioning on the values of |U
n
| and |V
n
|:
Pr
P
iV
n
Q
r
i
= x|A
n
= y
Pr
P
n
i=nΛ+1
D
i
= x|
P
n
i=nK+1
D
i
= y
=
P
L
max
κ=1
P
κ
λ=1
Pr(Λ = λ|K = κ) Pr
K = κ|
P
n
i=nK+1
D
i
= y
×Pr
P
n
nλ+1
D
i
|
P
n
i=nκ+1
D
i
= y
=
P
L
max
κ=1
P
κ
λ=1
Pr(Λ = λ|K = κ) Pr
K = κ|
P
n
i=nK+1
D
i
= y
×
Pr
D
(λ)
= x
Pr
D
(κλ)
= y x
Pr
D
(κ)
= y
. (55)
In expression (55) the probability Pr(K = κ|
P
n
i=nK+1
D
i
= y) is obtained by applying Bayes’
theorem:
Pr
K = κ|
P
n
i=nK+1
D
i
= y
=
Pr
D
(K)
= y|K = κ
Pr(K = κ)
Pr
D
(K)
= y
=
Pr
D
(κ)
= y
Pr(K = κ)
P
L
max
z=1
Pr
D
(z)
= y
Pr(K = z)
, (56)
24
while the probabilities Pr(Λ = λ|K = κ) are easily obtained from ϕ
κ,λ
, the joint density of Λ and
K.
Using this approximation for Pr
P
iV
n
Q
r
i
= x|A
n
= y
we can compute an approximation for
Pr(A = x) by finding the equilibrium distribution of the DTMC for A
n
. Then by using relation
(30) we obtain an approximation for the distribution of O as Pr(O = x) = Pr(A = ∆ x).
7. Numerical results
In this section we report on a numerical study to test the accuracy of the Markov Chain approxi-
mation that we propose. To this end a test bed of 1680 instances of problem P was created that is
a full factorial design of the parameter settings summarized in Table 7. The demand distributions
we used are mixtures of either two negative binomial or two geometric distributions. As such they
are discrete phase type distributions. These distributions were fitted on the first two moments
using the procedure suggested by Adan et al. (1996). For convenience we have included this fitting
procedure in the appendix. The different types of distributions for L are defined in Table 7.
Parameter settings
E[D] 25
c
2
D
1
4
,
1
2
, 1,
3
2
, 2
l
e
1, 2
E[L] 4, 8, 12
h 1
c 10, 20, 30, 40
γ
0
0.95, 0.98
Distribution type of L U1, U2, S1, S2, LS, RS, DET
Table 2: Test-bed of problem instances P
For each instance we performed the optimization by simulation. This was done by using recur-
sion (45) to determine the overshoot distribution. After a warm-up of 100 periods, recursion (45)
was computed until the width of 95% confidence intervals for E(O) and σ(O) was less than 1% of
the respective point estimates. Then we applied theorem 6.4 to find the DIP (S
sim
e
, S
sim
r
) to be
optimal with respect to simulation.
We also performed the optimization with our approach involving approximate Markov Chains,
which found the DIP (S
MC
e
, S
MC
r
) to be optimal with respect to the Markov chain approxima-
tion. Note that in literature optimization of the dual-index policy is always done using simulation
25
Pr(L = x)
Distribution Type \x E[L] 2 E[L] 1 E[L] E[L] + 1 E[L] + 2
U1 (uniform1) 0
1
3
1
3
1
3
0
U2 (uniform2)
1
5
1
5
1
5
1
5
1
5
S1 (symmetric1) 0
1
4
2
4
1
4
0
S2 (symmetric2)
1
10
2
10
4
10
2
10
1
10
LS (left skewed) 0
4
10
3
10
2
10
1
10
RS (right skewed)
1
10
2
10
3
10
4
10
0
DET (deterministic) 0 0 1 0 0
Table 3: Distribution types for L
(Kiesm¨uller, 2003, Scheller-Wolf et al. 2003, Veeraraghavan & Scheller-Wolf, 2008, Sheopuri et al.,
2010, Klosterhalfen et al. 2010) and so the solutions obtained through the simulation procedure
are the best solutions currently known. All computations were coded in MATLAB and executed
on a PC with an Intel-M 1.6 GHz processor with 768 MB of RAM.
Since both methods determine the total cost function and modified fill-rate with different meth-
ods, they are not directly comparable. Therefore, we evaluated the total cost and modified fill-rate
for both solutions using simulation. The simulation was run such that the width of 95%-confidence
interval for cost was less than 1% of the point estimate. As measures of optimality for the ap-
proximate Markov chain approach, we considered the relative deviation from the optimal DIP with
respect to simulation
C
=
C
sim
S
MC
r
, S
MC
e
C
sim
S
sim
r
, S
sim
e
C
sim
(S
sim
r
, S
sim
e
)
· 100%
and the absolute deviation from the target modified fill-rate
δ
γ
= γ
sim
S
MC
r
, S
MC
e
γ
0
.
Figure 7 shows a scatter-plot of
C
versus δ
γ
for the 240 problem instances with deterministic
lead times. Note that solutions that are optimal with respect to simulation would lie on the non-
negative x-axis. Thus solutions in the first quadrant outperform simulation solutions in that they
have lower costs while still meeting/exceeding the required service level. Solutions in the third
quadrant are dominated by simulation solutions while solutions in the second and fourth quadrant
are either more costly or do not meet the required service level. Note however that all service
level deviations are within 1% which is also the simulation tolerance and that negative relative cost
26
Figure 3: Quality of approximate optima with respect to simulation for deterministic lead times
deviations are also within the simulation tolerance. Thus the performance of the solutions found
are not statistically distinguishable from the simulation solutions.
Figure 7 shows a scatter-plot of
C
versus δ
γ
for the 1440 problem instances with stochastic
lead times. For stochastic lead times there is a tendency to find solutions that are more expensive
than simulation solutions at an increase in service relative to the target level. An explanation of the
superior performance of the approximation for deterministic lead times over stochastic lead times
is to be found in the fact that the approximation for deterministic lead times is based on limiting
results for and = 1, while for stochastic lead times they are based on limiting results
only for . However most solutions for instances with stochastic lead times are still very
close to the origin so that the approximation is usually very tight.
Since no instances differed significantly from the required service level, we investigated what
typified instances with a large relative cost deviation from the simulation optimum. To this end we
tabulated average minimum and maximum relative cost deviations from simulation solutions (∆
C
)
for the entire test bed for all problem parameters in Table 4.
We see that the approximation improves as demand variability increases. This is a convenient
property because dual sourcing is a way to buffer demand variability. Our approximation also
27
Figure 4: Quality of approximate optima with respect to simulation for stochastic lead times
becomes more accurate when the emergency lead time increases. This is in line with expectation
because holding cost can also be written as hE[(S
e
+ O D
(l
e
+1)
)
+
], from which we see that
the demand distribution (which we know exactly) influences holding cost more when l
e
is large.
Accuracy also increases when the expedition premium goes up. This is because expediting becomes
less attractive when c goes up, so that ∆ becomes larger and our approximation works better. That
our approximation becomes less accurate as E[L] increases can be explained again by inspecting the
holding cost hE[(S
e
+ O D
(l
e
+1)
)
+
]. The contribution of O, which we know only approximately,
compared to D
(l
e
+1)
becomes smaller when E[L] decreases. This is because ∆ (and therefore also O)
increases with E[K] = E[L] (by Little’s law). The target service level and different non-stochastic
distributions for L do not influence accuracy much. We do see as before that the approximation
performs much better when lead times are deterministic.
Computational times for our approximation are also much shorter than for the simulation based
procedure. For this test-bed the optimization method based on the approximation was on average
70 times faster than the simulation based method for problem instances with deterministic lead
times and on average 35 times faster for problem instances with stochastic lead times. For the
instances with deterministic lead times the optimization time of our procedure was always within 3
28
Relative cost deviation from simulation optimum [∆
C
]
c
2
D
avg. min max
1
4
2.13 -5.36 6.51
1
2
1.05 -2.78 4.30
1 0.32 -3.25 2.97
3
2
0.40 -1.10 2.07
2 0.06 -2.37 1.52
l
e
avg. min max
1 0.80 -3.25 6.51
2 0.76 -5.36 6.36
c avg. min max
10 1.35 -5.36 6.40
20 0.84 -2.06 6.51
30 0.59 -3.25 5.95
40 0.35 -4.10 5.95
γ
0
avg. min max
0.95 0.79 -4.10 6.51
0.98 0.77 -5.36 6.01
E[L] avg. min max
4 0.03 -5.36 4.72
8 0.95 -2.37 6.09
12 1.37 -1.56 6.51
Distribution type avg. min max
U1 0.88 -3.25 5.27
U2 0.98 -4.10 5.86
S1 1.01 -2.78 6.40
S2 1.13 -2.36 6.26
LS 1.06 -1.23 6.51
RS 0.90 -5.36 5.42
DET -0.48 -2.37 1.48
Total avg. min max
0.78 -5.36 6.51
Table 4: Quality of approximate solutions for different problem parameters
seconds while the computation time for the simulation approach was between 72 and 111 seconds.
For instances with stochastic lead times our approach always had a computation time within 15
seconds while the simulation based approach had a computation time between 98 and 183 seconds.
29
8. Conclusion and directions for future research
In this paper we presented two models. The first model deals with the dual-index policy for a
single stage dual sourcing inventory system facing stochastic demand with deterministic lead times
controlled by the dual-index policy. Our main contributions here are to (i) provide an alternate
and insightful proof of the separability result that reduces the optimization of the DIP to two
one-dimensional optimization problems and (ii) provide an approximate evaluation method of the
dual-index policy using Markov Chains based on limiting results that does not require simulation,
thus making optimization more efficient.
The second model we presented was a generalization of the first by allowing regular lead times
to be stochastic. In this situation we (i) defined a dual-index policy with mild informational
requirements on the realizations of regular lead times; (ii) proved that the same separability result
holds as for the model with deterministic lead time and (iii) developed an approximate evaluation
method using Markov Chains based on limiting results again making optimization much more
efficient.
In an extensive numerical study we showed that the approximations we suggest perform very
well in finding a close to optimal dual-index policy and are faster by at least an order of magnitude.
The research in this paper can be extended in several important ways. The most obvious and
possibly useful extension is to define and analyze the dual-index policy for multi-echelon inventory
systems. Consider for example a serial supply chain. Clark & Scarf (1960) showed that base-stock
policies are optimal for this system and that the optimal base-stock levels can be obtained by
successively solving newsvendor equations. This decomposition result relies on the fact that all
stock points in a serial supply chain face the same demand process. When the most downstream
stock point is the only stock point with two sources, this property is retained. In that case finding
the optimal echelon-DIP should be a straightforward task using the results in this paper.
When stock points other than the most downstream stock point have two sources the property
that each stock point essentially faces the same demand process is not preserved, because some
of the demand is ordered via the second source. Inventory control for this type of system is an
interesting new research direction. Perhaps a model closer to that of Lawson & Porteus (2000)
where there are different shipping modes between stock points could lead to tractability here.
Other research directions may generalize/extend the model to accommodate stochastic emer-
gency lead times, capacitated order quantities and fixed emergency ordering costs.
30
Acknowledgements
The authors thank the associate editor and two anonymous referees for their helpful comments
and suggestions. They also thank Jan van Doremalen and Geert-Jan van Houtum for helpful
discussions.
Most of this work was done while the first author was doing an internship at consultants in
quantitative methods.
References
[1] Adan, I.J.B.F., van Eenige, M.J.A., Resing, J.A.C. (1996) Fitting discrete distributions on the
first two moments, Probability in the Engineering and Informational Sciences, 9, 623-632
[2] Allon, G., van Mieghem, J.A. (2008) Global dual sourcing: tailored base surge allocation to
near and offshore production, Management Science, 56(1), 110-124
[3] Arslan, H., Ayhan, A., Olsen, T.V. (2001) Analytic models for when and how to expedite in
make-to-order systems, IIE Transactions. 33, 1019-1029
[4] Barankin, E.W. (1961) A delivery-lag inventory model with an emergency provision (the single-
period case), Naval Research Logistics Quarterly, 8, 285-311
[5] Chiang, C. (2007) Optimal control policy for a standing order inventory system, European
Journal of Operational Research, 182, 695-703
[6] Clark, A.J., Scarf, H. (1960) Optimal policies for a multiechelon inventory problem, Manage-
ment Science, 6(4), 475-490
[7] Feng, Q., Sethi, S.P., Yan, H., Zhang, H. (2006a) Optimality and nonoptimality of the base-
stock policy in inventory problems with multiple delivery modes, Journal of Industrial and
Management Optimization, 2(1), 19-42
[8] Feng, Q., Sethi, S.P., Yan, H., Zhang, H. (2006b) Are base-stock policies optimal in inventory
problems with multiple delivery modes?, Operations Research, 54(4), 801-807
[9] Fu, K., Hsu, V.N., Lee,C.Y. (2008) Optimal component acquisition for a single-product, single-
demand assemble-to-order problem with expediting, forthcoming in Manufacturing & Service
Operations Management
31
[10] Fukuda, Y. (1964) Optimal policies for the inventory problem with negotiable leadtime, Man-
agement Science, 10(4), 690-708
[11] Gallego, G., Jin, Y., Muriel, A., Zhang, G., Yildiz, V. (2007) Optimal ordering policies with
convertible lead times, European Journal of Operational Research, 176, 892-910
[12] Gaukler, G.M.,
¨
Ozer,
¨
O., Hausman, W.H. (2008) Order Progress Information: Improved Dy-
namic Emergency Ordering Policies, Production and Operations Management, 17(6), 599-613
[13] Hoen, K.M.R., G¨ull¨u, R., van Houtum, G.J., Vliegen, I.M.H. (2010) A simple and accurate
approximation for the order fill rates in lost-sales assemble-to-order systems, International
Journal of Production Economics in press
[14] Lawson D.G., Porteus, E.L. (2000) Multi-stage inventory management with expediting, Oper-
ations Research, 48(6), 878-893
[15] Kiesm¨uller, G.P. (2003) A new approach for controlling a hybrid stochastic manufacturing/re-
manufacturing system with inventories and different lead-times, European Journal of Opera-
tional Research, 147, 62-71
[16] Kosterhalfen, S., Kiesm¨uller, G.P., Minner, S. (2010) A comparison of the constant-order and
dual-index policy for dual sourcing, International Journal of Production economics forthcoming
[17] Minner, S. (2003) Multiple supplier inventory models in supply chain management: A review,
International Journal of Production Economics, 81-82, 265-279
[18] Plambeck, E.M., Ward, A.R. (2007) Note: a separation principle for a class of assemble-to-
order systems with expediting, Operations Research, 55(3), 603-609
[19] Robinson, L.W., Bradley, J.R., Thomas, L.T. (2001) Consequences of order crossover under
order-up-to inventory policies, Manufacturing & Service Operations Management, 3(3), 175-
188
[20] Rosenshine, M., Obee, D., (1976) Analysis of a standing order system with emergency orders,
Operations Research, 24(6), 1143-1155
[21] Sethi, S.P., Yan, H., Zhang, H. (2003) Inventory models with fixed costs, forecast updates and
two delivery modes, Operations Research, 51(2), 321-328
32
[22] Scheller-Wolf, A., Tayur, S. (2009) Risk sharing in supply chains using order bands - analytical
results and managerial insights, International Journal of Production Economics, 121, 715-727
[23] Scheller-Wolf, A., Veeraraghavan, S.K., van Houtum, G.J.J.A.N. (2003) Effective dual sourc-
ing with a single index policy, Working paper Tepper School of Business, Carnegie Mellon
University, Pittsburgh
[24] Sheopuri, A., Janakiraman, G., Seshadri, S. (2010) New policies for the stochastic inventory
control problem with two supply sources, Operations Research 58(3), 734-745
[25] Song, J.S., Zipkin, P. (2009) Inventories with multiple supply sources and networks of queues
with overflow bypasses, Management Science 55(3), 362-372
[26] van Houtum, G.J. (2006) Multiechelon production/inventory systems: optimal policies, heuris-
tics and algorithms, Tutorials in Operations Research INFORMS 2006, 163-199
[27] van Houtum, G.J., Zijm, W.H.M. (2000) On the relation between cost and service models for
general inventory systems, Statistica Neerlandica, 54(2) 127-147
[28] Veeraraghavan, S., Scheller-Wolf, A. (2008) Now or later: a simple policy for effective dual
sourcing in capacitated systems, Operations Research, 56(4), 850-864
[29] Whittmore, A.S., Saunders, S.C. (1977) Optimal inventory under stochastic demand with two
supply options, SIAM journal of applied mathematics, 32(2), 293-305
[30] Yazlali,
¨
O., Erhun, F. (2009) Dual-supply inventory problem with capacity limits on order
sizes and unrestricted ordering costs, IIE Transactions, 41, 716-729
[31] Zhou, S.X., Chao, X. (2010) Newsvendor bounds and heuristics for serial supply chains with
regular and expedited shipping, Naval Research Logistics, 57, 71-87
Appendix: Fitting procedure of Adan et al. (1996)
Here we describe a procedure for fitting discrete distributions on the first two moment of a discrete
random variable that is due to Adan et al. (1996).
Let X be a random variable on N {0} with mean µ
X
and squared coefficient of variation c
2
X
and define a = c
2
X
1
X
. Then the discrete random variable Y matches the first two moments of
X if it is chosen as follows:
33
If
1
k
a
1
k