A Multiprocessor Server-Based Scheduler for Soft Real-Time Tasks with Stochastic Execution Demand

Abstract

We utilize a multiprocessor server-based approach to schedule a general class of soft real-time systems with stochastic execution demands, when bounded average-case tardiness is sufficient for schedulability. A key feature of the task model considered here is that the stochastic execution-time demands can have arbitrary amounts of dependence within pre-specified time intervals of bounded length. This is an important practical step forward from requiring complete independence of execution times between successive jobs of the same task. Our main result does not require the scheduler to know the execution time of each job in advance, and requires only average-case utilization to be bounded by the number of processors. This constraint is mild compared to constraints on worst-case utilization because in multiprocessor systems, worst-case execution times may be orders of magnitude higher than average-case execution times.

Full text

Page 1

A Multiprocessor Server-Based Scheduler for
Soft Real-Time Tasks with Stochastic Execution Demand∗
Alex F. Mills
Department of Statistics and Operations Research
University of North Carolina
James H. Anderson
Department of Computer Science
University of North Carolina
Abstract
We utilize a multiprocessor server-based approach to sched-
ule a general class of soft real-time systems with stochas-
tic execution demands, when bounded average-case tardi-
ness is sufficient for schedulability. A key feature of the task
model considered here is that the stochastic execution-time
demands can have arbitrary amounts of dependence within
pre-specified time intervals of bounded length. This is an im-
portant practical step forward from requiring complete inde-
pendence of execution times between successive jobs of the
same task. Our main result does not require the scheduler
to know the execution time of each job in advance, and re-
quires only average-case utilization to be bounded by the
number of processors. This constraint is mild compared to
constraints on worst-case utilization because in multiproces-
sor systems, worst-case execution times may be orders of
magnitude higher than average-case execution times.
1Introduction
Previous work on scheduling soft real-time workloads has fo-
cused almost exclusively on allocating processing capacity to
jobs based on deterministic worst-case execution times. This
is a particular impediment in the implementation of soft real-
time systems, where some deadline tardiness is acceptable
and therefore the pessimistic assumption that every job may
require its worst-case execution time is unnecessary.
In multiprocessor systems, Leontyev and Anderson
showed that a number of global scheduling algorithms en-
sure bounded tardiness without utilization loss [8]; thus, soft
real-time workloads for which bounded tardiness is sufficient
can be supported on such systems. This result extended an
earlier proof by Devi and Anderson that showed the same
of the global earliest-deadline-first (GEDF) scheduling algo-
rithm [4]. In these results, utilizations are defined by assum-
ing worst-case execution costs.
In previous work, we questioned whether using a deter-
ministic worst-case execution time to compute utilization al-
ways makes sense in the realm of soft real-time systems [13].
∗Work supported by NSF grants CNS 0834270, CNS 0834132, and CNS
1016954; ARO grant W911NF-09-1-0535; AFOSR grant FA9550-09-1-
0549; and AFRL grant FA8750-11-1-0033.
The actual worst-case execution time of a task may be ob-
served so infrequently that it may not be worth dedicating
such a large amount of processing time to that task if bounded
tardiness is acceptable. Moreover, timing analysis tools may
add additional pessimism to the calculation of worst-case ex-
ecution times. These problems are exacerbated on a multipro-
cessor, where the worst-case scenario may be even less likely
but even more costly. In [13], we showed that processing
capacity could be allocated based on average-case execution
times, with the result that the expected (mean) tardiness of a
task is bounded when the system is scheduled using GEDF.
Unfortunately, the expected tardiness bound we derived still
depends on worst-case execution times, and the task model
also requires job execution times to be completely indepen-
dent of one another.
In this paper, we define a task model for soft real time
systems with stochastic execution demands (SRT-SED). SRT-
SED generalizes the model used in [13] and also other types
of task models, including the sporadic task model and a
stochastic or deterministic multiframe task model. We ana-
lyze SRT-SED task systems when scheduled on a system of
simple sporadic servers. Each server is a deterministically-
specified sporadic soft real-time task. By applying existing
analysis of soft real-time tasks that comprise the server sys-
tem, we prove a tardiness bound that has two special cases: a
deterministic, worst-case tardiness bound (when the underly-
ing system has worst-case utilization bounded by the number
of processors), and an expected, or average-case, tardiness
bound (when the underlying system has average-case utiliza-
tion bounded by the number of processors). The average-case
tardiness bound also yields a bound on the quantiles or per-
centiles of the tardiness distribution.
In this work, a job may only run when its task’s server
executes, and jobs that do not complete by the time their ex-
ecution budget runs out “steal” from the budget of the next
job of the same task. These ideas are not new in the real-
time systems literature, as the server scheme we use is similar
to bandwidth reservation. For example, Abeni and Buttazzo
used a bandwidth reservation strategy to schedule soft real-
time task systems on a uniprocessor, where successive jobs
of the same task have independent randomly distributed ex-

Page 2

ecution times [1]. However, only the case where worst-case
utilization is less than one was considered. Similarly, Brandt
et al.developed the resource allocation/dispatching (RAD)
model for uniprocessors, where tasks could have different
worst-case execution times and periods from job to job; the
RAD model led to the development of the rate-based-earliest-
deadline scheduler [3]. However, the RAD model assumes
that for real-time tasks, these parameters (e.g, worst-case ex-
ecution times) are known from job to job, in order to check
whether it is feasible to increase or decrease a task’s utiliza-
tion. Baruah et al.showed that constant bandwidth servers
(CBS) could be implemented on a multiprocessor to provide
a quality-of-service guarantee for each task [2], and Pelli-
zoni and Caccamo subsequently incorporated the reclaim-
ing of unused resources in a multiprocessor CBS [15], but
only hard real-time tasks with worst-case execution times
were considered in these research efforts. Finally, Leontyev
and Anderson developed a multiprocessor bandwidth reser-
vation scheme that encapsulates both hard and soft real-time
tasks [9], but again only for tasks with worst-case execution
times.
In this paper, we present a novel analysis of tardiness when
a server-based scheduling algorithm is used to schedule task
systems fitting the SRT-SED model: in particular, such task
systems may be over-provisioned in the worst case; to our
knowledge, this possibility has not been considered in any
previous work on multiprocessors. Unlike the papers men-
tioned above, our analysis does not place any constraint on
worst-case utilization, does not require the scheduler to know
the execution time of the job being scheduled, and does not
require worst-case execution times to be known or even to
exist (the only information needed is average-case execution
times and variance in execution times).
This paper also differs significantly from [13], where jobs
of every task could potentially be affected by a single job with
a much longer-than-average execution time. In this paper,
because the servers have a limited budget in each period, only
jobs of the same task can be affected by such overruns. There
are potential disadvantages in this case, but the result is that
a much more general class of soft real-time systems can be
analyzed. Moreover, unlike in [13], the expected tardiness
boundsdonotdependonworst-caseexecutiontimes—infact,
tasks that do not have a worst-case execution time can still be
scheduled.
Main Contribution.
We derive expected response time
bounds for tasks whose execution costs are stochastic, un-
der the scenario where those tasks are executed on simple
sporadic servers. The expected response time bound can be
applied to determine expected tardiness bounds and proba-
bilistic deadlines. Our task system specification is general
enough that certain types of dependence can be modeled
within its framework. Our tardiness bounds are applicable
if the average-case total utilization is less than the system’s
capacity, even if the system is over-utilized in the worst case
or does not have a worst case. We also separate the determin-
istic tardiness analysis from the stochastic analysis in such
a way that if better deterministic tardiness analysis becomes
available in the future, the results of this paper can immedi-
ately be applied without having to repeat any of the stochastic
analysis.
Organization.
The remainder of this paper is organized as
follows. In Section 2, we give a formal definition of the
system model. Section 3 consists of a number of technical
results that deal with the uncertainty in a stochastic system.
Section 4 derives the multiprocessor tardiness bounds. These
bounds are the main results of the paper. In Section 5, we dis-
cuss considerations necessary for the implementation of our
scheduling approach, and in Section 6, we remark on possible
uses and future directions for our model.
2System Model
We consider a system τ = {τ1,τ2,...,τn} consisting of n
tasks. A task is a possibly infinite sequence of jobs. Jobs are
demands for processing time, each with an associated dead-
line. Jobs of the same task must be executed sequentially and
in FIFO order. However, in this paper, we do not assume that
successive jobs of the same task are identical.
We consider the above system under different combina-
tions of the following assumptions, which are illustrated be-
low via several examples.
Assumption G. Assume that for each task τi ∈ τ, we can
find a set of time points {ti,0,si,1,ti,1,si,2,ti,2,...}
(whichmaybefiniteorcountablyinfinite)andaconstant
pisuch that for all j ≥ 1 the following four conditions
hold:
• ti,j− si,j≤ pi,
• si,j+1− si,j≥ pi,
• no job of τiis released in [ti,j−1,si,j), and
• no job of τireleased in [si,j,ti,j) has a deadline
later than ti,j.
Assumption D. Denote the total amount of processing time
demanded by jobs of τireleased in [si,j,ti,j) by Xi,j.
There exists a constant eisuch that for all tasks τi∈ τ,
and for all j = 1,2,..., Xi,j≤ ei.
Assumption S. Define Xi,jas above. For all tasks τi∈ τ,
the sequence {Xi,j,j = 1,2,...} is a sequence of in-
dependent and identically distributed random variables
with mean ¯ eiand variance σ2
i.
We will assume that Assumption G (a general assumption
about the analyzed SRT-SED task system) holds throughout
the paper. When Assumption D (the deterministic case) or
Assumption S (the stochastic case) are needed, this will be
stated explicitly. Note that the major contribution of this pa-
per is the set of results for the stochastic case; the results for
the deterministic case are included as a basis for comparison

Page 3

and for the sake of completeness; determinism is always a
special case of a stochastic system, namely, when variance is
zero.
To illustrate the types of task systems that are special cases
of SRT-SED, we provide a few examples.
Sporadic Task System.
Consider the typical constrained-
deadline sporadic task system where each τi∈ τ has period
Piand worst-case execution time Ci. We can see that this
system satisfies Assumptions G and D by setting si,jto be
the release time of the jth job of sporadic task τi, setting ti,j
to be the deadline of the jth job of τi, setting pi= Pi, and
setting ei= Ci.
Sporadic Stochastic Task System.
tem introduced in [13], where each τi ∈ τ has a period Pi
and an execution time that is independent and identically dis-
tributed with mean ¯ eiand variance σ2
task system makes no assumption that there is a worst-case
execution time, so Assumption D is not satisfied; however,
we can see that this system satisfies Assumption G by setting
si,jto be the release time of the jth job of sporadic task τi,
setting ti,jto be the deadline of the jth job of τi(which must
be implicit or constrained), and setting pi = Pi. Because
the execution times of successive jobs are independent and
identically distributed random variables, and only one job is
released between si,j−1and si,j, for all j, ¯ eiand σ2
Assumption S.
Multiframe Task System.
Consider a task system where
successive jobs of τicycle deterministically through Kidif-
ferent execution modes, as in [14]. Let Pidenote the period
of the cycle, and let all relative deadlines be constrained to
at most Pi. Furthermore, suppose that each execution mode
is characterized by a worst-case execution time Ci,k,k =
1,2,...,Ki. We can see that this system satisfies Assump-
tions G and D by setting si,j to be the release time of the
(jKi)th job (that is, the release time of the first job in the cy-
cle), setting ti,jto be the deadline of the ((j + 1)Ki− 1)th
job of τi(that is, the deadline of the last job in the cycle), set-
ting piequal to Pi, and setting ei=?Ki
Stochastic Multiframe Task System.
frame system like that above, but where each execution mode
is characterized by a different random execution time (that
is, the execution time is drawn from a different distribution
for each mode), such that the execution time in mode k in
one cycle is independent of the execution time in any mode
in any other cycle. Note that execution times may be depen-
dent within a cycle; for example, the execution time in mode
k may be correlated with the execution time in mode k − 1
within the same cycle.
We can see that this system satisfies Assumption G by set-
ting si,jto be the release time of the (jKi)th job (that is, the
release time of the first job in the cycle), setting ti,j to be
the deadline of the ((j + 1)Ki− 1)th job of τi(that is, the
Consider the task sys-
i. The definition of this
isatisfy
k=1Ci,k(that is, the
worst-case execution time of the entire cycle).
Consider a multi-
deadline of the last job in the cycle), and setting piequal to
Pi. To show that this system satisfies Assumption S, note that
the assumption of independence between cycles guarantees
that the total execution time of one cycle is independent of
the total execution time of any other cycle, and the fact that
execution times in each mode are identically distributed guar-
antees that the total execution time of the cycle is identically
distributed. Furthermore, the variance of the total execution
time of the cycle can be written as a function of the variance
of the execution time of each mode and the covariance be-
tween execution times of different modes (see, e.g., [16] for
details).
2.1Independence
In order for Assumption S to be satisfied, execution time de-
mands in successive [s,t) time intervals must be independent
from one another. Many applications of real-time systems
have jobs with clear dependence on one another. Such de-
pendence may result explicitly from the type of application
(e.g., in a video decoding application, if a frame takes a long
time to decode because it is part of a scene with significant
movement, then its successor is more likely to also take a
long time to decode) or they may result from characteristics
of the system (e.g., if a job runs quickly due to a warm cache,
then its successor is more likely to also run quickly if it re-
quires access to the same set of data). We note that our system
definition does not require execution times of each job to be
independent of one another. However, allowing arbitrary de-
pendence would result in completely intractable analysis—in
fact, complete independence is commonly assumed in many
queueing applications (for example, independence of succes-
sive service times) for the purpose of tractability. The sys-
tem model presented in this paper lies somewhere between
the most true-to-life model (which would be intractable), and
the somewhat unrealistic model that calls for independence of
every job’s execution time (which was used in prior work on
real-time systems [1, 13]). Moreover, because the intervals in
which dependence is permitted can be set by the user, there
is an interesting design trade-off: longer intervals result in a
moreaccuratemodelattheexpenseoflargertardinessbounds
(as we shall see). In some applications, these intervals may
occur naturally. In other cases, these intervals may have to
be imposed by the system designer: e.g., for dependencies
arising from cache affinity, the beginning of a new indepen-
dent interval corresponds in practical terms to a cold cache.
In the latter case, the system designer would need to calculate
the execution time demand by assuming a cold cache at the
beginning of each interval.
2.2Simple Sporadic Servers
The scheduling scheme we will use in this paper involves a
server abstraction. Each task τi ∈ τ will run on a unique
simple sporadic server Ti, which will carefully control the
amount of time that τiis allowed to execute. This will there-

Page 4

fore ensure that worst-case execution times do not appear in
the average-case tardiness bound.
Let T = {T1,T2,...,Tn} be a system of simple sporadic
servers. Each simple sporadic server Tihas period piand
execution budget bi. We will discuss conditions that must
be satisfied by biin the next section. The budget of Tiis
replenished whenever Tiis eligible and backlogged. Tiis
eligibleifandonlyifithasneverhaditsbudgetreplenishedor
at least pitime units have elapsed since its last replenishment
(recallthatpiwasdefinedinAssumptionG).Tiisbacklogged
if and only if there is pending work of τi. We call Ti,jthe jth
instance of Ti. The replenishment time of Ti,jis denoted by
Ri,j. The deadline of Ti,j, denoted Di,j, is the earliest time at
which Ti,j+1could be replenished, which is Di,j= Ri,j+pi
(i.e., server deadlines are implicit).
The budget of Tiis consumed whenever it has the high-
est priority according to the scheduling algorithm in use. For
example, if the scheduling algorithm is GEDF, then the bud-
get of Tiis consumed whenever it has one of the m earliest
deadlines, where m is the number of processors.
2.3
Figure 1(b) shows an example with sporadic stochastic tasks
τ1, with period 5 and execution cost drawn from some distri-
bution with mean 2, and τ2, with period 3 and execution cost
drawn from some distribution with mean 0.75. In this model,
we let τi,jdenote the jth job of task τi. τ1,1is released at
time 0 and has execution cost 4, τ1,2is released at time 6.3
and has execution cost 1.5, and τ1,3is released at time 11.3
and has execution cost 2. τ2,1is released at time 0 and has
execution cost 0.8, and τ2,2is released at time 3 and has exe-
cution cost 1.7. The schedule is not shown after time 13. As
we discussed above, the sporadic stochastic task model satis-
fies the assumptions required for task systems in our paper.
Figure 1(a) shows two servers. T1corresponds to τ1. It
has period 5.0 (the same as τ1), and budget 3. T2 corre-
sponds to τ2. It has period 3 and budget 1. We can ver-
ify that T is schedulable by EDF on a uniprocessor because
3/5 + 1/3 ≤ 1. We will use the example given in Figure
1 to illustrate some important properties of the considered
scheduling approach.
Scheduling Example
Initial Replenishment.
time 0 because they are both eligible (having never been re-
plenished before) and backlogged (because both τ1and τ2
release jobs at that time). The deadlines are set to 5 (for T1)
and 3 (for T2).
Consumption Rule.
Budgets are consumed according to
how the server instances of T are scheduled. In the example,
T is scheduled using EDF on a uniprocessor. For example,
T2begins consuming its budget first at time 0 because T2,1
has a higher priority (earlier deadline) than T1,1.
Idleness.
T is scheduled without regard for idleness in
τ. Although τ2,1finishes executing at 0.8, T2continues its
Both servers are replenished at
1 , 1
1T
0
2 , 1
3 , 1
4
8
12
0
4
8
12
2T
0
4
8
12
0
4
8
12
2
1
1 , 2
2 , 2
ExecutionSuspension
(a)
(b)
Figure 1: Example servers and sporadic stochastic tasks on
a uniprocessor. (a) For servers T1and T2, ↑ denotes replen-
ishment time and ↓ denotes deadline; the budget is shaded
for each server. T1and T2are scheduled using EDF. (b) For
the sporadic stochastic tasks τ1and τ2, ↑ denotes release time
and ↓ denotes deadline; the actual execution times are shaded,
while suspensions are shown in white.
consumption, even though this means that the processor is
idle. This prevents server instances from experiencing self-
suspensions. Such self-suspensions potentially could be al-
lowed, at the expense of needing to impose utilization con-
straints to bound server tardiness [11]. However, we do not
consider that possibility in this paper.
Suspensions.
sumed, so τ1,1suspends its execution. At time 5, T1is eligi-
ble, so its budget is replenished, even though τ1,2has not yet
been released. τ1,1resumes executing and continues execut-
ing until time 6, when it completes.
At time 4, the budget of T1,1has been con-
Replenishment Rule.
only when it is eligible and backlogged. Therefore, a replen-
ishment will not necessarily occur at the next server deadline.
For example, T1becomes eligible for replenishment at time
10 but is not replenished until time 11.3 because no job of τ1
can execute until that time.
The budget of a server is replenished

Page 5

Table 1: Summary of Notation
Notation
[si,j,ti,j)
Definition
jth interval during which dependence
is allowed for τi
execution requirement for τiover [si,j,ti,j)
budget of server Ti
jth instance of Ti
jth replenishment time of Ti
deadline of Ti,j
largest index k such that Ri,k< ti,j
remaining work of τiwith deadline ≤ ti,j
not completed by Di,(j)+1
Xi,j
bi
Ti,j
Ri,j
Di,j
(j)
Wi,j
2.4
We define the remaining work process Wi,jto be the amount
ofworkofτiduetojobswithdeadlineatmostti,jthathasnot
completed by the time the budget of server instance Ti,(j)+1
has been exhausted, where (j) = max{k : Ri,k < ti,j}.
That is, (j) is the index of the last server instance that replen-
ishes Tibefore ti,j. We will use the parenthetical function in
conjunction with all other server-related notation. For exam-
ple, Ri,(j)is the replenishment time of Ti,(j)and Di,(j)is the
deadline of Ti,(j).
To maintain consistency in the analysis, we define Wi,0=
0 because no work of τ is released prior to ti,0, which is de-
fined to be time 0.
Note that in general Ti,(j)is not the jth replenishment of
Ti, because replenishments of Timay occur more frequently
than time points in the set {si,j,j = 0,1,...}. This will
happen if the interval (ti,j−1,si,j) is long enough and Tiis
backlogged during that interval.
A full summary of notation is given in Table 1.
Remaining Work Process
Lemma 1. Any job of τiwith a deadline no later than ti,j
completes no later than the actual time at which the server
instance Ti,kcompletes, where k = (j) + 1 + ?Wi,j/bi?.
Proof. Take an arbitrary job with deadline no later than ti,j.
Either τi,j completes by the time Ti,(j)+1completes, or it
does not. If it does, the result is immediate, because Ti,(j)+1
completes before any later instance of Ti(in particular, Ti,k).
Otherwise, there is some work remaining for our job when
Ti,(j)+1completes. The amount of such work is no more than
Wi,j, because Wi,jby definition includes all work due to jobs
of τireleased prior to ti,jthat did not complete by the time
Ti,(j)+1completed.
Because all work of τiis completed sequentially, we can
guarantee that the remaining work of τi,jis completed once
Tihas executed for at least Wi,jadditional time units follow-
ing the completion of Ti,(j)+1. Because each instance of Ti
completes bitime units of work on τiwhen there is pending
work of τi, this means that the remaining work of τi,jwill
complete no later than the time when Ti,kcompletes.
3
This section consists of technical results that lead to a bound
on the expected value of the remaining work process.
Analysis of the Remaining Work Process
Lemma 2. The remaining work process satisfies
Wi,0= 0
Wi,j= max{0,Wi,j−1+ Xi,j− Yi,jbi}, for j = 1,2,...
(1)
(2)
where Yi,j is the number of times Ti is replenished in
[ti,j−1,ti,j).
Proof. First, observe that prior to the first release, there is
clearly no remaining work, so (1) holds.
Next, by definition, Xi,jis the amount of work that may
add to the remaining work that is counted in Wi,jbut not in
Wi,j−1, because it is the amount of work that is demanded
by τi in [si,j,ti,j) (remember that no jobs are released in
[ti,j−1,si,j)). Finally, observe that by the time Ti,(j)+1com-
pletes, either an additional Yi,jbiunits of work will have been
allocated to τi, or there will be no more remaining work of τi;
hence, either we subtract Yi,jbifrom the remaining work and
are left with positive remaining work, or else we subtract all
the remaining work, and are left with zero (justifying the max
operation).
Define the process Zi,jas follows:
Zi,0= 0
Zi,j= max{0,Zi,j−1+ Xi,j− bi}, for j = 1,2,.... (4)
The next lemma relates this new process with the remain-
ing work process. Note that when we describe the relation-
ship between two random quantities as A ≤ B, we mean “the
probability that A is at most B is 1.”
(3)
Lemma 3. Wi,j≤ Zi,jfor all i,j.
Proof. First, Wi,0= Zi,0= 0. Now, proceed by induction.
Suppose that Wi,j−1 ≤ Zi,j−1. Observe that if Wi,j > 0
then Yi,j≥ 1, because Tiwill be replenished at least once in
[ti,j−1,ti,j) when there is any remaining work of τi. Thus
Wi,j= max{0,Wi,j−1+ Xi,j− Yi,jbi}
≤ max{0,Wi,j−1+ Xi,j− bi}
≤ max{0,Zi,j−1+ Xi,j− bi}
= Zi,j.
By induction, the result holds.
Lemma 4. If E(Xi,j) < bi, then {Zi,j,j = 1,2,...} has a
limit Zi, as j → ∞.
Proof. In [10], analytical results are presented concerning
processes having the form of Zi,j. It is shown there that as
j increases, the distribution of Zi,jhas a limit if and only if
E(Xi,j− bi) < 0.