Page 1

Passage-time Computation and Aggregation Strategies for Large

Semi-Markov Processes

Marcel C. Guenther, Nicholas J. Dingle∗, Jeremy T. Bradley, William J. Knottenbelt

Department of Computing, Imperial College London, 180 Queen’s Gate, London SW7 2BZ, United Kingdom

Abstract

High-level semi-Markov modelling paradigms such as semi-Markov stochastic Petri nets and pro-

cess algebras are used to capture realistic performance models of computer and communication

systems but often have the drawback of generating huge underlying semi-Markov processes. Ex-

traction of performance measures such as steady-state probabilities and passage-time distributions

therefore relies on sparse matrix–vector operations involving very large transition matrices. Pre-

vious studies have shown that exact state-by-state aggregation of semi-Markov processes can be

applied to reduce the number of states. This can, however, lead to a dramatic increase in matrix

density caused by the creation of additional transitions between remaining states. Our paper

addresses this issue by presenting the concept of state space partitioning for aggregation.

Aggregation of partitions can be done in one of two ways. The first is to use exact state-by-state

aggregation to aggregate each individual state within a partition. However, we discover that this

approach still causes matrix density problems, albeit on a much smaller scale compared to non-

partition aggregation. A second approach to the aggregation of partitions, and the one presented in

this paper, is atomic partition aggregation. Inspired by a technique used in passage-time analysis,

this collapses a whole partition into a small number of semi-Markov states and transitions.

Most partitionings produced by existing graph partitioners are not suitable for use with our atomic

partition aggregation techniques, and we therefore present a new deterministic partitioning method

which we term barrier partitioning. We show that barrier partitioning is capable of splitting very

large semi-Markov models into a number of partitions such that first passage-time analysis can be

performed more quickly and using up to 99% less memory than existing algorithms.

Key words: Semi-Markov processes, Aggregation, Passage-time analysis

1. Introduction

Semi-Markov processes (SMPs) are expressive tools for modelling a wide range of real-life systems.

The state space explosion problem, however, hinders the analysis of large finite SMPs as it does

of many stochastic and functional modelling disciplines. One approach to addressing this problem

is to use aggregation techniques to remove single states or groups of states and aggregate their

temporal effect into the remaining states. Many techniques exist in the Markovian domain for exact

and approximate aggregation (e.g. lumpability [17], aggregation/disaggregation [11], aggregation

of hierarchical models [10]) but to date analogous work on semi-Markov aggregation algorithms

∗Corresponding author

Email addresses: mcg05@doc.ic.ac.uk (Marcel C. Guenther), njd200@doc.ic.ac.uk (Nicholas J. Dingle),

jb@doc.ic.ac.uk (Jeremy T. Bradley), wjk@doc.ic.ac.uk (William J. Knottenbelt)

Preprint submitted to ElsevierSeptember 25, 2009

Page 2

has been very limited. In prior work [5, 8], we presented an aggregation algorithm for semi-Markov

processes which operates on each state individually. Our analysis in [8] suggests that the primary

limitation of this technique is that the computational cost and memory requirements become very

large as increasing numbers of states are aggregated and the transition matrices representing the

SMP consequently gets less sparse.

In this paper, we present a number of novel approaches for overcoming the aggregation problem.

Central to these is the concept of partitioning the state space, and we begin by considering different

partitioning methods (initially inspired by those previously used for parallel sparse matrix–vector

multiplication) and evaluating their suitability for our state-by-state aggregation algorithm. We

demonstrate that by partitioning the state space in this way and then using the state-by-state

aggregation algorithm on the separate partitions, as opposed to applying it directly to an unpar-

titioned state-space, we can reduce the computational cost and memory requirements of our exact

aggregation approach.

However, even when applied to partitions of the semi-Markov process there is a central drawback of

exact state-by-state aggregation. Although the result of the process is an aggregated and smaller

state space, the intermediate steps can actually create more state transitions (and hence require

more storage and computational effort) than were present in the original unaggregated state space.

Inspired by our prior work on iterative passage-time analysis in SMPs [9], we therefore present

atomic partition aggregation to overcome this limitation. This does not require each state in the

partition to be aggregated in turn, but instead effectively calculates the passage-time distribution

across an entire partition and combines this with the state holding time distributions of relevant

states outside the partition. As partitioning techniques suitable for parallel sparse matrix–vector

multiplication do not produce partitions suitable for the application of atomic aggregation, we

also introduce a new barrier partitioning strategy which is better suited. We demonstrate how

this enables passage-time analysis to be conducted in less time and using up to 99% less memory

than before.

The remainder of this paper is organised as follows. Section 2 summarises background theory on

the calculation of passage times in semi-Markov processes from [9], and also summarises our state-

by-state aggregation technique [8]. Section 3 then introduces the concept of performing aggregation

on partitions of the state space, and discusses the importance of the order in which partitions are

chosen to be aggregated. Section 4 then presents our novel atomic aggregation approach where

whole partitions are aggregated by means of a passage-style analysis. Section 5 presents the

barrier partitioning technique and evaluates the improvements in the memory and time required

to analyse large semi-Markov models, that barrier partitioning offers. Finally, Section 6 concludes

and suggests directions for future work.

2. Background

2.1. Semi-Markov Processes

Semi-Markov Processes (SMPs) are an extension of Markov processes which allow for generally

distributed sojourn times [19, 20]. Although the memoryless property no longer holds for state

sojourn times, at transition instants SMPs still behave in the same way as Markov processes (that

is to say, the choice of the next state is based only on the current state) and so share some of their

analytical tractability.

Consider a Markov renewal process {(χn,Tn) : n ≥ 0} where Tnis the time of the nth transition

(T0= 0) andχn∈ S is the state at the nth transition. Let the kernel of this process be:

R(n,i,j,t) = IP(χn+1= j,Tn+1− Tn≤ t |χn= i)

2

Page 3

for i,j ∈ S. The continuous time semi-Markov process, {Z(t),t ≥ 0}, defined by the kernel R, is

related to the Markov renewal process by:

Z(t) =χN(t)

where N(t) = max{n : Tn≤ t}, i.e. the number of state transitions that have taken place by time

t. Thus Z(t) represents the state of the system at time t. We consider only time-homogeneous

SMPs in which R(n,i,j,t) is independent of n:

R(i,j,t)=

=

IP(χn+1= j,Tn+1− Tn≤ t |χn= i)

pijHij(t)

for any n ≥ 0

where pij= IP(χn+1= j |χn= i) is the state transition probability between states i and j and

Hij(t) = IP(Tn+1−Tn≤ t |χn+1= j,χn= i), is the sojourn time distribution in state i when the

next state is j. An SMP can therefore be characterised by two matrices P and H with elements

pijand Hijrespectively.

2.2. Iterative Passage-time Algorithm

In this section we define the first passage-time random variable used throughout the paper. We also

summarise from [9] an iterative algorithm for calculating first passage-time density in semi-Markov

processes.

From now on, we consider a finite, irreducible, continuous-time semi-Markov process with N states

{1,2,...,N}. Recalling that Z(t) denotes the state of the SMP at time t (t ≥ 0) and that N(t)

denotes the number of transitions which have occurred by time t, the first passage time from a

source state i at time t into a non-empty set of target states?j is defined as:

Pi?j(t) = inf{u > 0 : Z(t + u) ∈?j,N(t + u) > N(t),Z(t) = i}

For a stationary time-homogeneous SMP, Pi?j(t) is independent of t:

Pi?j= inf{u > 0 : Z(u) ∈?j,N(u) > 0,Z(0) = i}

(1)

This formulation of the random variable Pi?japplies to an SMP with no immediate transitions. If

such transitions are present, then the passage time can be stated as:

Pi?j= inf{u > 0 : N(u) ≥ Mi?j}

(2)

where Mi?j= min{m ∈ Z Z+:χm∈?j |χ0= i} is the transition marking the terminating state of

the passage.

Pi?jhas an associated probability density function fi?j(t). The Laplace transform of fi?j(t), Li?j(s),

can be computed by means of a first-step analysis. That is, we consider moving from the source

state i into the set of its immediate successors?k and must distinguish between those members of

?k which are target states and those which are not. This calculation can be achieved by solving a

set of N linear equations of the form:

?

where r∗

by:

r∗

0

Li?j(s) =

k/ ∈?j

r∗

ik(s)Lk?j(s) +

?

k∈?j

r∗

ik(s): for 1 ≤ i ≤ N

(3)

ik(s) is the Laplace–Stieltjes transform (LST) of R(i,k,t) from Section 2.1 and is defined

?∞

ik(s) =

e−stdR(i,k,t)(4)

3

Page 4

Eq. (3) has matrix–vector form Ax = b, where the elements of A are general functions of the

complex variable s. For example, when?j = {1}, Eq. (3) yields:

We now describe an iterative algorithm for generating passage-time densities that creates succes-

sively better approximations to the SMP passage-time quantity Pi?jof Eq. (1) [9]. We approximate

Pi?jas P(r)

occur starting from state i and ending in any of the states in?j. We calculate P(r)

and then numerically inverting [1, 2, 3] its Laplace transform L(r)

1

0

0

...

0

−r∗

1 − r∗

−r∗

12(s)

22(s)

32(s)

...

N2(s)

···

···

···

...

···

−r∗

−r∗

−r∗

1N(s)

2N(s)

3N(s)

...

NN(s)

−r∗

1 − r∗

L1?j(s)

L2?j(s)

L3?j(s)

...

LN?j(s)

=

r∗

r∗

r∗

11(s)

21(s)

31(s)

...

r∗

N1(s)

(5)

i?j, for a sufficiently large value of r, which is the time for r consecutive transitions to

i?j

by constructing

i?j(s).

Recall the semi-Markov process Z(t) of Section 2.1, where N(t) is the number of state transitions

that have taken place by time t. We formally define the rth transition first passage time to be:

P(r)

i?j

= inf{u > 0 : Z(u) ∈?j,0 < N(u) ≤ r,Z(0) = i}

(6)

which is the time taken to enter a state in?j for the first time having started in state i at time 0

and having undergone up to r state transitions.

If we have immediate transitions in our SMP model (as in Eq. (2)) then the rth transition first

passage time is:

P(r)

i?j

This is because as the firing of an immediate transitions results in zero time being spent in the

state in which it was enabled, it is not meaningful to talk about the SMP being in a particular

state at a particular time. Instead, we count the transitions which have happened so that we may

reason about the order in which they have occurred.

= inf{u > 0 : Mi?j≤ N(u) ≤ r}

P(r)

i?j

component of the vector:

is a random variable with associated Laplace transform L(r)

i?j(s). L(r)

i?j(s) is, in turn, the ith

L(r)

?j(s) =

?

L(r)

1?j(s),L(r)

2?j(s),...,L(r)

N?j(s)

?

representing the passage time for terminating in?j for each possible start state. This vector may

be computed as:

L(r)

?j(s) = U

?

I + U?+ U?2+ ··· + U?(r−1)?

pq(s) and U?is a modified version of U with elements

e?j

(7)

where U is a matrix with elements upq= r∗

u?

otherwise. The initial multiplication with U in Eq. (7) is included so as to generate cycle times

for cases such as L(r)

vector e?jhas entries ek?j= δk∈?j, where δk∈?j= 1 if k is a target state (k ∈?j) and 0 otherwise.

From Eq. (1) and Eq. (6):

pq= δp?∈?jupq, where states in?j have been made absorbing. Here, δp?∈?j= 1 if p ?∈?j and 0

ii(s) which would otherwise register as 0 if U?were used instead. The column

Pi?j= P(∞)

i?j

and thus

Li?j(s) = L(∞)

i?j

(s)

This can be generalised to multiple source states?i using, for example, a normalised steady-state

vector α calculated from π, the steady-state vector of the embedded discrete-time Markov chain

4

Page 5

Fig. 1. Reducing a complete 4 state graph to a complete 3 state graph.

(DTMC) with one-step transition probability matrix P = [pij,1 ≤ i,j ≤ N], as:

?

The row vector with components αkis denoted by α. The formulation of L(r)

αk=

πk/?

j∈?iπj

if k ∈?i

otherwise0

(8)

?i?j(s) is therefore:

L(r)

?i?j(s)=

αL(r)

?j(s)

=(αU + αUU?+ αUU?2+ ··· + αUU?(r−1))e?j

r−1

?

=

k=0

αUU?ke?j

(9)

The sum of Eq. (9) can be computed efficiently using sparse matrix–vector multiplications with a

vector accumulator, µr=?r

In practice, convergence of the sum L(r)

particular r and s-point:

k=0αU?k. At each step, the accumulator (initialised as µ0= αU) is

updated as µr+1= αU + µrU?.

?i?j(s) =?r−1

|Re(L(r+1)

where ε is chosen to be a suitably small value, say ε = 10−16.

k=0αUU?kcan be said to have occurred if, for a

?i?j

(s) − L(r)

?i?j(s))| < ε

and

|Im(L(r+1)

?i?j

(s) − L(r)

?i?j(s))| < ε

(10)

2.3. Exact State Aggregation

In order to control the state space explosion which occurs when generating the state transition

matrix for a semi-Markov process, we have previously developed an exact aggregation algorithm

that acts on the semi-Markov state space directly [5, 8]. The aim is to apply the aggregation before

performing any passage-time or transient analysis and thus reduce the calculation time required

to solve the system of linear equations shown in Eq. (5).

5